At 11:28 AM 7/30/98 -0400, Scott Gifford wrote: I have been using a packet shaper (non commercial product) since 4/96 and have recently been reviewing/testing a number of commercial products on the market. I will ignore the options and features every product has but rather focus on those things to look at when checking these boxes: 1) alerts: a nice feature to have that no commercial product has yet is an alert feature when a bandwidth limiting policy has been exceeded. Example: You limit incoming icmp to 100kb and are suddenly hit by a Smurf eating up 2Mb/sec. You want an alert sent to the NOC via email or some other method. Or you have policy limiting single stream outgoing UDP to 256kb and suddenly see 4Mb/sec. You definitely want an alarm going off somewhere. We have found that email alerts are invaluable. Packeteer doesn't have it. 2) Bandwidth manipulation: some products do buffering, some play with the TCP window. You want both. Packeteer has both. 3) Bypass: you want a hardware and software bypass in the event something goes wrong. This is black box sitting in the data path. You now have a strip of boxes: firewall->packetshaper->router (and perhaps more). If something doesn't work, you don't want to start recabling to bypass the packet shaper. Software bypass via a Web interface is a must, and a hardware bypass is also critical. Packeteer has both. 4) Signon management: In a large ISP operation, many users will need to have access to modify the rules and policies. You will want a system that has individual user signon rather than a single user/pswd. Packeteer has only single user/pswd so therefore it is impossible to track who has done what changes. 5) Topflows: In RMON there is the concept of Top 10. When the network slows down, you want the ability to see a list of the Top n users consuming bandwidth in the past 1 minute/10 minutes/hour. Packeteer does not have that capability. 6) Flows: When a vendor says their box supports T3, one has to check a little deeper and determine the number of max flows allowed. For example, Allot (www.allot.com) supports a T3 box, but tops out at 5000 concurrent flows. Packeteer supports 20,000 concurrent flows. Current realtime numbers on my beta box show 2400 flows consuming 3.3Mb, and often 8000-9000 flows consuming 6.5Mb of bandwidth. Based on those numbers, I will hit 20,000 flows long before I hit 20Mb of bandwidth. I do not think the packet shaper vendors have much of an idea in the ISP market as to the large number of flows involved. They know corporate networks. 7) Platform: Look at the OS platform. Packeteer using a proprietary OS, others may package Linux or NT. None have done any OS hardening on the system so it is best to run something like ISS against the packet shaper to determine what security holes exist. Imagine you start using a packet shaper in production only to have the hackers hack it and set their own super-duper policies. 8) Audit trail: The product should have the ability to have some sort of syslog based on modifications done by personnel. Packeteer has something that is not accessible via the web and is more for debugging. 9) Filters & CPU: Look for products that can support CIDR notation and not the long masks. Packeteer doesn't support CIDR notation masking for filters. Some limit the size of the filter (also called policy, access- list, pipe rule, etc.). Packeteer limits it to 16 lines. The larger the filter the slower the box. Test it out. 10) ToD: All boxes have the ability to control based on source IP, destination IP and port. Not all have the ability to control based on time of day. Suppose you want incoming news to be limited to 128kb during the day but open it up from 2-8am to 800kb. Packeteer has a line command called "schedule". Look for GUI's to do this. 11) Flow limiting: sometimes you want the ability to not only control bandwidth but also the number of flows. Example: you allow IP phone via the Internet via port xxxx but want to guarantee 8kb/sec per flow, but you only have 64kb of bandwidth allocated for that port. You want to have the ability to state that a maximum of 8 flows can use that virtual pipe and the 9th won't get the service. Packeteer says they have this ability but I have not verified it. 12) Graphs: you want the ability for realtime graphs for each policy so you can see how your rule changes have affected the bandwidth. Packeteer has this capability. 13) Multihoming: this is the one *every* vendor fails on and is perhaps the one we need. These packet shaper boxes assume either one outgoing line from the router, or if there are multiple lines - that they are load balanced (NReality - www.nreality.com places their box on the FR line itself after the router - and last I checked only support FR). But in most ISP cases, you are multihomed with 3-4 outgoing lines - which are not fully balanced. Suppose one line is 90% and the others are 30%. The packet shaper can't see any of this and therefore the policy rules are not perfect. The line that is 90% satuarted is hearing 4,000 nets via BGP. How do you get that routing info the packet shaper? None have some sort of BGP import and when I tell them I want to import 52,000 prefixes and build policy rules based on that - they look at me like I am from Mars. In the near future we will be exploring this avenue for Internet-2 in Israel. 14) Transparent proxy: remember that line of boxes? firewall->packet shaper->router? Now add in a transparent proxy. Ugh. Look for a vendor that will include a transparent proxy capability in their box. I wouldn't be suprised if Alteon and Packeteer were to merge. These kind of mergers have to happen. Checkpoint already has packet shaping in their firewall via an addon product called Floodgate. Cisco bought up Classdata (www.classdata.com) so expect to see more of these capabilities in firewalls and routers. If you have read articles in Network World, Data Communications, etc. on this topic you will not find this level of detail there. This only comes with actually using and testing these boxes. Scott, these products do work and are not snake oil. I trust this helps you somewhat. -Hank
I was wondering if anybody out there has had any experience with "Packet Shapers," which claim to be able to limit traffic to a particular host, host+port, or host+port+url without dropping packets. They apparently watch traffic flows and keep track of connections, then dink around with TCP window size information to slow traffic down if needed.
The one we have been looking at is called "The Packeteer" (http://www.packeteer.com), but I have seen a few others (one of which had a really good picture of a pig in the ad).
Do these things work as advertised? Is anybody actively using them inside their networks? I don't know enough down-and-dirty information about TCP to know if this should work, or if it's just a plastic shell filled with snake-oil...
Thanks for any information. Email if you like, I will post a summary if people are interested.
------Scott.
[..]
14) Transparent proxy: remember that line of boxes? firewall->packet shaper->router? Now add in a transparent proxy. Ugh. Look for a vendor that will include a transparent proxy capability in their box. I wouldn't be suprised if Alteon and Packeteer were to merge. These kind of mergers have to happen. Checkpoint already has packet shaping in their firewall via an addon product called Floodgate. Cisco bought up Classdata (www.classdata.com) so expect to see more of these capabilities in firewalls and routers.
Err, CLASS Data Systems has very little to do with that for now. Besides, you already have plenty of IP QoS mechanisms in cisco today, to implement various queuing and drop policies based IP TOS, as well as integrating technologies underneath IP to transfer IP QoS into the underlying transport. Most of what you guys talk about can be done on many IOS platforms today without buying additional hardware. Granted, you need cisco hardware (or a mainframe for an OS/390 IOS stack :), but either way you have the option of doing all that in a switch/router. The methods discussed so far are certainly not scalable for large service providers. Btw: DS-3, IMHO, does not count as real true broadband. T1's are becoming a fairly small pipe, especially in the advent of technologies such as DSL. Which makes DS-3 the next best thing up the ladder, short of buying a half a dozen or more DSL pipes. DS-3's are what T1's are getting to be. There is a lot of good lit out from all the major vendors on the subject of traffic shaping and engineering. Why introduce yet another box into the network? Cheers, Chris -- Christian Kuhtz, BellSouth Corp., Sr. Network Architect <ck@bellsouth.net> 1100 Ashwood Parkway, Atlanta, GA 30338 <ck@gnu.org>
On Fri, 31 Jul 1998, Christian Kuhtz wrote:
[..]
14) Transparent proxy: remember that line of boxes? firewall->packet shaper->router? Now add in a transparent proxy. Ugh. Look for a vendor that will include a transparent proxy capability in their box. I wouldn't be suprised if Alteon and Packeteer were to merge. These kind of mergers have to happen. Checkpoint already has packet shaping in their firewall via an addon product called Floodgate. Cisco bought up Classdata (www.classdata.com) so expect to see more of these capabilities in firewalls and routers.
Err, CLASS Data Systems has very little to do with that for now.
Besides, you already have plenty of IP QoS mechanisms in cisco today, to implement various queuing and drop policies based IP TOS, as well as integrating technologies underneath IP to transfer IP QoS into the underlying transport. Most of what you guys talk about can be done on many IOS platforms today without buying additional hardware. Granted, you need cisco hardware (or a mainframe for an OS/390 IOS stack :), but either way you have the option of doing all that in a switch/router.
The methods discussed so far are certainly not scalable for large service providers. Btw: DS-3, IMHO, does not count as real true broadband. T1's are becoming a fairly small pipe, especially in the advent of technologies such as DSL. Which makes DS-3 the next best thing up the ladder, short of buying a half a dozen or more DSL pipes. DS-3's are what T1's are getting to be.
There is a lot of good lit out from all the major vendors on the subject of traffic shaping and engineering. Why introduce yet another box into the network?
Not everyone has Cisco, and the QA on Cisco releases has not been the greatest. Bash in all of the above into IOS and how much you wanna bet that we will need version (19cc) before all the components work (i.e. FR, SNMP, HSSI, PPP, BGP, etc.) Each of us has slowed down in installing new versions in critical systems due to these numerous bugs and flaws. Cisco has WCCP but yet many people prefer Alteon or Intokomi (sp?). Ask yourself why. You are on the mark in regards to OC-3 being too small these days. None of these boxes scale to OC-12; neither do firewalls. Wouldn't be suprised if some Pluris or Juniper came out with some MPP box that can do all of the above. -Hank
Cheers, Chris
-- Christian Kuhtz, BellSouth Corp., Sr. Network Architect <ck@bellsouth.net> 1100 AshwoodParkway, Atlanta, GA 30338 <ck@gnu.org>
Hank Nussbacher
Not everyone has Cisco, and the QA on Cisco releases has not been the greatest. Bash in all of the above into IOS and how much you wanna bet that we will need version (19cc) before all the components work (i.e. FR, SNMP, HSSI, PPP, BGP, etc.) Each of us has slowed down in installing new versions in critical systems due to these numerous bugs and flaws.
I think some of this could be solved if cisco's release methodology was properly understood industry wide. It is somewhat unique. The CC train especially is different that way (it breaks a lot of constraints and boundaries otherwise designed to keep bugs out of the code in favor of getting near geekcode and feature rich stuff out there quickly).
Cisco has WCCP but yet many people prefer Alteon or Intokomi (sp?). Ask yourself why.
Well, given certain scenarios, WCCP is actually a significant strength, and Inktomi/etc's reliance on another vendors' L2/L4 switch does not only increase complexity of the solution (multiple vendors etc) but also tie in into backbones, for instance. Alteon to me is everything but a strength. WCCP is probably only in its infancy, and I have found that its functionality is overall not well understood. And the competitive market does its fair share to declassify WCCP as policy routing in a new gown etc etc -- no need for me to reproduce their marketing propaganda, just the stuff you'd expect in a market segment with heated (and overvalued) IPOs etc.
You are on the mark in regards to OC-3 being too small these days. None of these boxes scale to OC-12; neither do firewalls. Wouldn't be suprised if some Pluris or Juniper came out with some MPP box that can do all of the above.
Well, I think that I have all the above in our current platform, actually. Once Juniper actually has a product which we can get into the labs, I'd love to beat it up to see what it is worth and how good its Q&A is etc etc. No doubt, they are on the horizon, but not here yet. Unless perhaps someone from Juniper is listening and could get in touch with me to get this in here. Cheers, Chris -- Christian Kuhtz, BellSouth Corp., Sr. Network Architect <ck@bellsouth.net> 1100 Ashwood Parkway, Atlanta, GA 30338 <ck@gnu.org>
In message <005501bdbc4f$2e8665c0$0540d6d1@oreo.eng.bellsouth.net>, "Christian Kuhtz" writes:
shaper->router? Now add in a transparent proxy. Ugh. Look for a vendor that will include a transparent proxy capability in their box. I .. The methods discussed so far are certainly not scalable for large service providers. Btw: DS-3, IMHO, does not count as real true broadband. T1's ..
i guess its a matter of where you deploy them within your network. certainly, i wouldn't see deployment of transparent proxy boxes in the middle of a large backbone working particularly well -- both in terms of scaling issues, but perhaps more as a political issue (as recently shown by the testostorone-charged-thread in this list not-so- long-ago when digex put a netapp netcache transparently on some customers -- presumably via some vendors' L4 switch). where they _do_ win, however, is at the access-router-edge of your network. there are currently things that some vendors' boxes can do, that can't be done (yet) in IOS. .. and for some of them, they are scaling beyond a DS3 already. for one vendor's gear (alteon), for transparent proxying, i took it up to 2 x DS3's without it skipping a beat. i stopped there, as it is academic in this country (australia) to be going beyond that, for a pool of dialup customers of the largest dialup isp and the only cablemodem offering. i guess the point being that limitations were hit in the actual proxy-cache- boxes well before limitations were hit in layer-4 switch functionality. (biggest factor being: (total-transaction-time x trans-per-sec) > max-fd's). cheers, lincoln. (nb, yes, cisco do have WCCP - however, with it effectively being a proprietary protocol, and whilst some of the functionality is possible in policy-based-routing, you lose out big time when your cache isn't functioning and the router is still (blindly) feeding it all the http-flows).
In message <199807311225.WAA22845@interlink.com.au>, Lincoln Dale writes:
(as recently shown by the testostorone-charged-thread in this list not-so- long-ago when digex put a netapp netcache transparently on some customers -- presumably via some vendors' L4 switch).
d'oh. s/netapp netcache/inktomi traffic-server/ cheers, lincoln.
i guess its a matter of where you deploy them within your network.
Yep.
certainly, i wouldn't see deployment of transparent proxy boxes in the middle of a large backbone working particularly well -- both in terms of scaling issues, but perhaps more as a political issue
It actually is an architectural hazard, not just a cosmetical booboo.
(as recently shown by the testostorone-charged-thread in this list not-so- long-ago when digex put a netapp netcache transparently on some customers -- presumably via some vendors' L4 switch).
Yep. Inktomi via Alteon. And the latter is the former's biggest weakness in an otherwise fine product.
where they _do_ win, however, is at the access-router-edge of your network.
Yep. Do you really see the market in providing just bigger IP cores, though? Only in that assumption, all value add is at the edge. I happen to strongly disagree with that.
there are currently things that some vendors' boxes can do, that can't be done (yet) in IOS. .. and for some of them, they are scaling beyond a DS3 already.
True. But, can they manage hundreds of pipes across an infrastructure? Do they have the ability to tie into provisioning systems, billing systems etc? Nope.
for one vendor's gear (alteon), for transparent proxying, i took it up to 2 x DS3's without it skipping a beat. i stopped there, as it is academic in this country (australia) to be going beyond that, for a pool of dialup customers of the largest dialup isp and the only cablemodem offering.
i guess the point being that limitations were hit in the actual proxy-cache- boxes well before limitations were hit in layer-4 switch functionality. (biggest factor being: (total-transaction-time x trans-per-sec) > max-fd's).
cheers,
lincoln.
(nb, yes, cisco do have WCCP - however, with it effectively being a proprietary protocol, and whilst some of the functionality is possible in policy-based-routing,
Policy based routing is not what WCCP is about. But I am glad Inktomi's (or whoever) drill worked on ya :).
you lose out big time when your cache isn't functioning and the router is still (blindly) feeding it all the http-flows).
That's why WCCP is more than just policy routing, 'cause it doesn't do that. Don't get me wrong, Inktomi and Alteon is a fine product, having spent plenty of time with both products and with the Inktomi folks. But it doesn't fit the backbone integration bill as the flyers make you want to believe. Simply because not everything up there is Ether or has to ever traverse through it. I don't see infrastructure pick'n'chose, lowest common denominator networking (="best of breed") as a viable long term strategy. And I'm not getting paid to provide quick fixes ;). Cheers, Chris -- Christian Kuhtz, BellSouth Corp., Sr. Network Architect <ck@bellsouth.net> 1100 Ashwood Parkway, Atlanta, GA 30338 <ck@gnu.org>
In message <000a01bdbc95$d2bc59b0$2b0698cd@oreo.eng.bellsouth.net>, "Christian Kuhtz" writes:
where they _do_ win, however, is at the access-router-edge of your network.
Yep. Do you really see the market in providing just bigger IP cores, though? Only in that assumption, all value add is at the edge. I happen to strongly disagree with that.
i'm not stating that the value is only at the edge. i'm stating that i don't think it is realistic to be sticking transparent devices that hold detailed state information on flows in the core. there is enormous value in the core: some people get it right, some get it horribly wrong. but lets face it - the core of a networks primary function is to move packets fast, and move them well.
True. But, can they manage hundreds of pipes across an infrastructure? Do they have the ability to tie into provisioning systems, billing systems etc? Nope.
granted, they cannot appear on just any interface (serial/hssi/posip/atm/..) with just about any encapsulation - they are limited to fast/giga/ethernet - but this is also what i'm talking about deployment-at-edge. ... unless your border routers go posip/atm straight to your core routers? is there any reason why they can't look into provisioning/billing systems? its only a matter of some scripting glue . . . the main point i was trying to make is this one anyway:
i guess the point being that limitations were hit in the actual proxy-cache- boxes well before limitations were hit in layer-4 switch functionality. (biggest factor being: (total-transaction-time x trans-per-sec) > max-fd's).
that is, well before you hit limitations on L4-type-devices in your network, you'll hit limitations in the devices that accept the redirected packets _from_ the L4 devices. this, in its own right, makes deploying in the core unworkable (....for the time being....).
(nb, yes, cisco do have WCCP - however, with it effectively being a proprietary protocol, and whilst some of the functionality is possible in policy-based-routing,
Policy based routing is not what WCCP is about. But I am glad Inktomi's (or whoever) drill worked on ya :).
<shrug> its effectively dynamic-policy-based-routing on http flows. i say 'dynamic', since cache-farm-participants can join/leave the policy. if no members are left in the farm, packets go down their natural path. (.. and that isn't out of some other vendors' sales pitch either) cheers, lincoln.
Yep. Do you really see the market in providing just bigger IP cores, though? Only in that assumption, all value add is at the edge. I happen to strongly disagree with that.
i'm not stating that the value is only at the edge. i'm stating that i don't think it is realistic to be sticking transparent devices that hold detailed state information on flows in the core.
From a perspective within a TCP/IP core, a flow is a state generated by a stream of packets which have one or more identifying characteristics in common. If you want to do anything in terms of flows or indentifying IP CoS etc or perhaps gather statistics, you need to maintain some sort of state. Perhaps not to make switching decisions right away, but at least collect the state information and process it to then execute decisions down the road.
there is enormous value in the core: some people get it right, some get it horribly wrong. but lets face it - the core of a networks primary function is to move packets fast, and move them well.
I strongly disagree with that. See other email that I had just sent.
granted, they cannot appear on just any interface (serial/hssi/posip/atm/..) with just about any encapsulation - they are limited to fast/giga/ethernet - but this is also what i'm talking about deployment-at-edge. ... unless your border routers go posip/atm straight to your core routers?
They may. If you look down the road at the next generation access concentration, you may get exactly what you just described above. I don't want to restrict my core design _AND_ functionality implications based on my reliance on Ethernet technology. Ethernet is a cheap fast, mature local area pipe at the moment. Not a value add to my network.
is there any reason why they can't look into provisioning/billing systems? its only a matter of some scripting glue . . .
What I have in the back of my mind goes far beyond what you can achieve with scripting "glue". Sure, you could write a wallstreet trading system in Perl, but I wouldn't recommend it ;). How about backoffice systems beyond the point of providing a reference to devices in the network?
the main point i was trying to make is this one anyway:
i guess the point being that limitations were hit in the actual proxy-cache- boxes well before limitations were hit in layer-4 switch functionality. (biggest factor being: (total-transaction-time x trans-per-sec) > max-fd's).
that is, well before you hit limitations on L4-type-devices in your network, you'll hit limitations in the devices that accept the redirected packets _from_ the L4 devices.
Hmm.
this, in its own right, makes deploying in the core unworkable (....for the time being....).
You assume a particular technology/schematic model. I don't think I see that basis for your argument as a valid cornerstone of future network core technology. Cheers, Chris -- Christian Kuhtz, BellSouth Corp., Sr. Network Architect <ck@bellsouth.net> 1100 Ashwood Parkway, Atlanta, GA 30338 <ck@gnu.org>
On Fri, 31 Jul 1998, Lincoln Dale wrote:
(nb, yes, cisco do have WCCP - however, with it effectively being a proprietary protocol, and whilst some of the functionality is possible in policy-based-routing, you lose out big time when your cache isn't functioning and the router is still (blindly) feeding it all the http-flows).
If a Cache Engine goes down - the remainder of the farm will redistribute the load - if the farm is unresponsive the core router will stop redirecting port 80 until such a time as it can start communicating with a cache speaking WCCP once again. -- I am nothing if not net-Q! - ras@poppa.clubrich.tiac.net
participants (4)
-
Christian Kuhtz -
Hank Nussbacher -
Lincoln Dale -
Rich Sena