We are considering using Prolexic to 'defend' our Internet-facing network from DDOS attacks. Anyone have any known issues or word of warnings before we proceed? Chris Cunningham Network Engineering Secure Connectivity 704-427-3557 (Desk) 704-701-6924 (Cell) Samuel.Cunningham@Wellsfargo.com<mailto:Samuel.cunningham@wellsfargo.com> [X] This e-mail, including any attached files, may contain confidential and privileged information for the sole use of the intended recipient. Any review, use, distribution, or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive information for the intended recipient), please contact the sender by reply e-mail and delete all copies of this message.
On Oct 19, 2011, at 3:13 PM, <samuel.cunningham@wellsfargo.com> <samuel.cunningham@wellsfargo.com> wrote:
We are considering using Prolexic to 'defend' our Internet-facing network from DDOS attacks. Anyone have any known issues or word of warnings before we proceed?
They say that "When an attack is detected, our protection services are implemented within minutes. Upon activation, a Prolexic customer routes in-bound traffic to the nearest Prolexic scrubbing center where proprietary-filtering techniques, advanced routing, and patent-pending hardware devices remove bot traffic close to the source." You may want to ask them how near is "nearest" before you proceed. If their sensor is too far from your systems, simply sending all that traffic to the scrubbing center could be overkill. The last phrase suggests they use sink holing, though. -- PacketDam: a cost-effective software solution against DDoS http://www.packetdam.com
On Wed, Oct 19, 2011 at 8:46 AM, Vlad Galu <galu@packetdam.com> wrote:
They say that "When an attack is detected, our protection services are implemented within minutes. Upon activation, a Prolexic customer routes in-bound traffic to the nearest Prolexic scrubbing center where proprietary-filtering techniques, advanced routing, and patent-pending hardware devices remove bot traffic close to the source."
If that's true, that they have a technology that is so good they will only describe it as proprietary magic, that will efficiently scrub all bot traffic and not scrub legitimate traffic, and it really works every time, and they've persuaded you/made a convincing argument, i'd be thrilled. But probably there is no way to validate that it will actually work to your satisfaction against whatever sort of attack you face, before actually buying the service. If they are confident it is so great, they ought to be happy to either price your cost of the service based on its effectiveness or otherwise provide you a very good SLA, clearly stipulate what they can and can't handle, both in terms of type of attack, and volume of attack (realistically, there is some flood volume that will exceed _any_ service's capacity). Make sure they will waive fees, early termination fees at least, fees for "protection they failed to provide" at least, and give you a cancel option/way out, should their technology fail to be as effective as their marketing would have you believe, and make sure they have a burden of proof under the SLA to show their protection service worked properly after an incident, rather than you having to prove it did not. Clearly you would want to discuss the technical details with them and costs, whether some sort of subscription or per-incident; that protection services are only implemented "when activated", indicates there is cost or technical disadvantage during any time you choose to have "protection active" - -JH
We've dealt with these guys too too. There are lots of providers; I've used ones through ISPs and they can work well. Our only issue is that the ISP we were talking with only had XYZ Gb of mitigation, and Prolexic has a ton more capacity (in the hundreds of gigabits when I last checked). Prolexic is the go-to company for handling large-scale DDoSes. We haven't yet tried the service, but they've been extremely professional. Every time we're on the phone it's with engineers that know their stuff. Ultimately you're going to want to have a mix of internal mitigation and one or several providers if you're a big target. I doubt anyone is going to be perfect -- it's simply impossible. Heck, lots of the attacking "bots" are just spyware on legitimate users' PCs, so obviously they will get blocked. My personal experience is that when you're dealing with a DoS at the scale that you need Prolexic, there is simply no one else that can handle that level of traffic. -Andreas On Sat, Oct 22, 2011 at 6:22 PM, Jimmy Hess <mysidia@gmail.com> wrote:
They say that "When an attack is detected, our protection services are implemented within minutes. Upon activation, a Prolexic customer routes in-bound traffic to the nearest Prolexic scrubbing center where proprietary-filtering techniques, advanced routing, and
On Wed, Oct 19, 2011 at 8:46 AM, Vlad Galu <galu@packetdam.com> wrote: patent-pending hardware devices
remove bot traffic close to the source."
If that's true, that they have a technology that is so good they will only describe it as proprietary magic, that will efficiently scrub all bot traffic and not scrub legitimate traffic, and it really works every time, and they've persuaded you/made a convincing argument, i'd be thrilled. But probably there is no way to validate that it will actually work to your satisfaction against whatever sort of attack you face, before actually buying the service.
If they are confident it is so great, they ought to be happy to either price your cost of the service based on its effectiveness or otherwise provide you a very good SLA, clearly stipulate what they can and can't handle, both in terms of type of attack, and volume of attack (realistically, there is some flood volume that will exceed _any_ service's capacity). Make sure they will waive fees, early termination fees at least, fees for "protection they failed to provide" at least, and give you a cancel option/way out, should their technology fail to be as effective as their marketing would have you believe, and make sure they have a burden of proof under the SLA to show their protection service worked properly after an incident, rather than you having to prove it did not.
Clearly you would want to discuss the technical details with them and costs, whether some sort of subscription or per-incident; that protection services are only implemented "when activated", indicates there is cost or technical disadvantage during any time you choose to have "protection active"
- -JH
On 10/24/2011 1:54 PM, Andreas Echavez wrote:
obviously they will get blocked. My personal experience is that when you're dealing with a DoS at the scale that you need Prolexic, there is simply no one else that can handle that level of traffic.
Andreas, I think there are a lot of people on this list that would argue with that statement. As was mentioned earlier, AT&T, Verizon, and several others including Verisign have very ample networks capable of handling attacks just as large as Prolexic, if not bigger. One thing to note about Prolexic, where it stands out from some of the others is that it is a completely off-net solution. Many of the other offerings from folks like Verizon require you to have WAN circuits connected to their network in order to avail of such a service (in other words, they will only scrub that which would normally traverse their network on it's way towards your WAN interface). Others like Verisign have (smartly) adopted a similar model to that of Prolexic. They understand that requiring a physical connection into a provider's cloud is a monolithic approach (and certainly runs counter to today's mantra of offering up cloud-based services). Stefan Fouant JNCIE-SEC, JNCIE-SP, JNCIE-ER, JNCI Technical Trainer, Juniper Networks Follow us on Twitter @JuniperEducate
On Mon, Oct 24, 2011 at 3:29 PM, Stefan Fouant <sfouant@shortestpathfirst.net> wrote:
On 10/24/2011 1:54 PM, Andreas Echavez wrote:
obviously they will get blocked. My personal experience is that when you're dealing with a DoS at the scale that you need Prolexic, there is simply no one else that can handle that level of traffic.
Andreas,
I think there are a lot of people on this list that would argue with that statement. As was mentioned earlier, AT&T, Verizon, and several others including Verisign have very ample networks capable of handling attacks just as large as Prolexic, if not bigger.
One thing to note about Prolexic, where it stands out from some of the others is that it is a completely off-net solution. Many of the other offerings from folks like Verizon require you to have WAN circuits connected to their network in order to avail of such a service (in other words, they will only scrub that which would normally traverse their network on it's way towards your WAN interface).
Others like Verisign have (smartly) adopted a similar model to that of Prolexic. They understand that requiring a physical connection into a provider's cloud is a monolithic approach (and certainly runs counter to today's mantra of offering up cloud-based services).
but... often the cost of scrubbing includes the cost of transit to/from the remote provider, which is why 'cheapest' only counts for an entire process, NOT for 'lookie, I bought the service!'. either way, folks will learn one way or the other which works for them. -chris
Stefan Fouant JNCIE-SEC, JNCIE-SP, JNCIE-ER, JNCI Technical Trainer, Juniper Networks
Follow us on Twitter @JuniperEducate
On 10/24/2011 3:53 PM, Christopher Morrow wrote:
On Mon, Oct 24, 2011 at 3:29 PM, Stefan Fouant
but... often the cost of scrubbing includes the cost of transit to/from the remote provider, which is why 'cheapest' only counts for an entire process, NOT for 'lookie, I bought the service!'.
either way, folks will learn one way or the other which works for them.
I couldn't agree with you more - often times there are unintended costs, for example, the operational burden of moving your advertisements towards the provider who offers a scrubbing service... Also the more complex it is to use a particular service, the more likely you are to have indirect costs in terms of lost revenue during the outage. All of these things should be properly vetted well in advance, and the additional operational burden should also be factored into the pricing equation. Unfortunately, all too often these additional things aren't factored by the bean counters until it's too late. Stefan Fouant JNCIE-SEC, JNCIE-SP, JNCIE-ER, JNCI Technical Trainer, Juniper Networks Follow us on Twitter @JuniperEducate
Having used some of the largest solutions, I do disagree. After quickly searching google for Verisign, I could find a few documents that claim they have ~350Gb of capacity. On Prolexic's website, they claim to have the largest <http://www.prolexic.com/why-prolexic/index.html> total mitigation capacity at 375Gb. Now if you're talking about upstream providers (ATT/Verizon), even if your upstream mitigates the traffic, do you really N+1 redundancy during an attack? Do the providers have an SLA guaranteeing mitigation within a certain timeframe? Finally, and most importantly to us, was how much do they charge per attack, or if it a flat "insurance" type agreement where they block unlimited attacks. Total capacity certainly isn't the most important factor, but a sane pricing policy certainly was. -Andreas On Mon, Oct 24, 2011 at 12:29 PM, Stefan Fouant < sfouant@shortestpathfirst.net> wrote:
On 10/24/2011 1:54 PM, Andreas Echavez wrote:
obviously they will get blocked. My personal experience is that when
you're dealing with a DoS at the scale that you need Prolexic, there is simply no one else that can handle that level of traffic.
Andreas,
I think there are a lot of people on this list that would argue with that statement. As was mentioned earlier, AT&T, Verizon, and several others including Verisign have very ample networks capable of handling attacks just as large as Prolexic, if not bigger.
One thing to note about Prolexic, where it stands out from some of the others is that it is a completely off-net solution. Many of the other offerings from folks like Verizon require you to have WAN circuits connected to their network in order to avail of such a service (in other words, they will only scrub that which would normally traverse their network on it's way towards your WAN interface).
Others like Verisign have (smartly) adopted a similar model to that of Prolexic. They understand that requiring a physical connection into a provider's cloud is a monolithic approach (and certainly runs counter to today's mantra of offering up cloud-based services).
Stefan Fouant JNCIE-SEC, JNCIE-SP, JNCIE-ER, JNCI Technical Trainer, Juniper Networks
Follow us on Twitter @JuniperEducate
On Mon, Oct 24, 2011 at 6:46 PM, Andreas Echavez <andreas@livejournalinc.com> wrote:
certain timeframe? Finally, and most importantly to us, was how much do they charge per attack, or if it a flat "insurance" type agreement where they block unlimited attacks.
for verizon the 'time to mitigate' is gated on you sending a community for the route, how fast can you do that? the charge is a flat cost/month - it was 3250/month at one point (list price).
Total capacity certainly isn't the most important factor, but a sane pricing policy certainly was.
right, that was my point about the off-net services. -chris
On Oct 24, 2011, at 10:54 AM, Andreas Echavez wrote:
Prolexic is the go-to company for handling large-scale DDoSes. We haven't yet tried the service, but they've been extremely professional.
Not sure I understand your post. You claim Prolexic are the go-to-guys, and extremely professional… but you haven't used them? I would agree with Stephan's response as well, some of the other providers have as much capacity to deal with attacks (Verisign, Neustar, etc). And it's not about what's "stated" on their marketing slicks, it's about actual capacity, architecture, and "clue." Prolexic has a long (early) history of DDoS mitigation, and I have no reason do doubt they are any worse than they used to be but if you haven't used them, it's just conjecture. I'd be interested to know whom you have experience with and what size of attack you were able to mitigate with them (not being pedantic, but looking for real-world examples and all). -b
On Mon, Oct 24, 2011 at 4:45 PM, Brett Watson <brett@the-watsons.org> wrote:
On Oct 24, 2011, at 10:54 AM, Andreas Echavez wrote:
Prolexic is the go-to company for handling large-scale DDoSes. We haven't yet tried the service, but they've been extremely professional.
Not sure I understand your post. You claim Prolexic are the go-to-guys, and extremely professional… but you haven't used them?
I would agree with Stephan's response as well, some of the other providers have as much capacity to deal with attacks (Verisign, Neustar, etc). And it's not about what's "stated" on their marketing slicks, it's about actual capacity, architecture, and "clue."
Agreed, however our point of contention was that no other providers were willing to write SLAs based on service delivery time. We've used Verizon's service and it took nearly 10-12 hours coordinating with their NOC to get the service up and running, then over a week of troubleshooting packet sizes and so forth to finally get the system working properly. Unfortunately the only way for us to test Prolexic is to come under attack. In the meantime, the provisioning, engineering team, and everyone else has been fantastic. I'm not trying to push one provider over another -- we've just had good communication. Someone with less frequent or smaller attacks may find better value in another service. Prolexic's stated current network capacity is 375Gb. They have *claimed* that they will have 500Gb total by next year.
Prolexic has a long (early) history of DDoS mitigation, and I have no reason do doubt they are any worse than they used to be but if you haven't used them, it's just conjecture.
That's all I'm really saying here. It's been a good experience so far -- but only time will tell. Most of these *providers* are just using Arbor networks equipment and a fat pipe. It generally all works the same. Unfortunately it's not a simple task to test several hundred gigabytes of mitigation capacity.
I'd be interested to know whom you have experience with and what size of attack you were able to mitigate with them (not being pedantic, but looking for real-world examples and all).
We were able to mitigate a 20Gb attack through VZB. It was concerning because their total network capacity is 80Gb across ~4 PoPs. Unfortunately we had the issues above, combined with a lot of billing confusion on their part. They asked us to pay more for no reason whatsoever because we really need to *upgrade* our tier to the 1Gb service from the 500Mb (what does that mean)? This conversation with their sales team followed the somewhat large attack stated above. When asked "does the 1Gb tier mean 1Gb of clean traffic, or that you block 1Gb of DDoS", they couldn't answer our question. Anyhow take everything with a grain of salt. Our experience could differ vastly than others, and this isn't mean I have anything against Verizon or anyone else.
-b
-Andreas
On Wed, Oct 19, 2011 at 9:13 AM, <samuel.cunningham@wellsfargo.com> wrote:
We are considering using Prolexic to 'defend' our Internet-facing network from DDOS attacks. Anyone have any known issues or word of warnings before we proceed?
you appear to be an ATT customer (and qwest) ATT has a dos-mitigation solution, it works well enough... why not just use theirs? it's guaranteed to be cheaper (in the case of an actual attack) as compared to prolexic (and closer to your actual website/presence... (since they have a scrubbing center in stl) -chris
participants (7)
-
Andreas Echavez -
Brett Watson -
Christopher Morrow -
Jimmy Hess -
samuel.cunningham＠wellsfargo.com -
Stefan Fouant -
Vlad Galu