On Oct 14, 2008, at 12:08 PM, Scott Doty wrote:
First, the good news: so far, the NANOG conference has been very valuable and content-rich, covering a lot of issues that need to be discussed. For that, I am grateful.
But now, the bad news(?): Maybe it's just me & my paranoia, but do I detect an inkling of "murk spam" going on with some presentations?
I fully agree with you -- some talks are thinly (or not so thinly) veiled attempts to convince you to buy a vendor's shiny, new solution. There are a large number of reasons for this, and the Program Committee works hard (and I think is doing a great job) to limit the amount of sales pitch but A: there are a limited number of talks and B: many vendors are unable to resist trying to spin their product. I suggest that if you have a topic that you would like to present (and will keep it sales free) you resent it to the PC. I *do* however disagree with you that this happened in the talks to which you are referring...
Because there seems to be a fundamental misunderstanding, either on my part, or the part of certain vendors: I'm hear to discuss ideas & freely share them, and they are here to discuss (it would seem) their products.
Once again, great -- please submit a talk to the PC and they will review it. The PC is always looking for good talks...
Sometimes both goals coincide, and that is fine...but...
When a vendor at the security BOF starts showing documents that are "company confidential", and trying to whip up a climate of fear, that we should all deploy their product in front of our recursive name servers, i get this funny feeling that I am being "murk spammed".
Hmmm... The vendor that you are referring to provides authoritative DNS for many domains (and, at least some of them I view as "important", meaning that I would prefer a correct response!). Yes, I am sure that he would be happy to have you as a customer and, yes, this is feature that differentiates his company, but I did not get the impression AT ALL that he was trying to sell his service, but rather provide better service to his existing customers, even going so far as to provide free devices to people who run large recursive resolvers. This helps both his existing customers (who, yes, will be more likely to continue using him), but, more importantly helps me as an end user feel a little comfortable that the page that I am getting is the correct page...
Perhaps that is my own perspective (& paranoia?), but I found the CERT gentleman's call to monitor icmp backscatter on our authoritative nameservers far more informative -- and open.
But I was disappointed with two vendors and their presentations: the first had the tactic of saying "DNSSEC is the actual solution" when asked about why their product would be necessary...completely ignoring the fact that their proprietary "interim solution" was by no means the only way to prevent cache poisoning attacks.
I may be mistaken, but I didn't get the impression that he believed that his solution was the only one -- he repeatedly pointed out that DNSSEC is the correct solution and this his solution does not solve all of the problems that DNSSEC would -- however, DNSSEC is FAR from being fully deployed.
Indeed, I would daresay it isn't the best, either by a BCP perspective, or a cost analysis perspective.
To put a finer point on this, i should say that i found myself discomforted by a presentation suggesting that I should put their proprietary appliances between my recursive name servers & the Net, and I am grateful that Mr. Vixie stood up and said that there are other ways of dealing with the problem.
Hmmm.. We must have VERY different recollections -- I don't remember him mentioning how much this would cost, other than that he would be give away some to the biggest wins first. Without knowing how much these widgets will be, it is not possibly to do a cost comparison, but don't discount just how expensive engineering time is, and just how hard it is to find competent DNS folks able to deploy something else. I have chatted with many people about the state of their DNS infrastructure -- many people don't care, many people DO care but just don't have the cycles to properly maintain it, many have weird internal politics around them, and many just don't have the knowledge. Some of these are hard to solve, the lack of knowledge is probably the easiest, so I would welcome any how0-to, etc guides that would feel like writing....
Then there was the gentleman with the DDOS detection/mitigation appliance, who flipped through several graphs, which were intended to show the number of each type of attack. It's unfortunate that there wasn't more time for questions, because I really wanted to ask why "http GET" and "spidering" attacks weren't listen on their graphs...more on that in a second.
Hmmm, probably some of this is my fault, I am largely responsible for the agenda -- this was my first tie doing this an I suspect that I tried to fit too many talks into too little time. If there had been more time Danny might have covered their collection methodology (but, I need to warn you that that would probably have involved some information that *could* be construed as "This is what differentiates us" and would have been construed as sales, but whatever...). The information that was presented is part of a very well know report that gets published (but in a more executive format) and he (apparently incorrectly) assumed that the BOF audience would already be aware of how the information is collected and some of the benefits and short comings of their collection methodology. Once agin, probably my fault that he didn't have enough time to go though how the data is collected, but if he had, most of the audience would have bored out of their minds and they already know this and the rest would have felt like they were being sold to...
Fortunately, said vendor had a table at "beer and gear", so I was able to talk with one of their representatives -- and learned that they have just as much trouble with automatic detection of attacks designed to look like a "slashdotting"...which cleared up the mystery as to why it wasn't on the graphs.
Because this is a real problem: anybody, with sufficient knowledge & preparation can vandalize _anybody's_ network. Showing me a graph that ping floods happen all the time doesn't impress me -- what would impress me is going over the actual methods, algorithms (and heuristics?) used in these attack mitigation appliances.
Ok, now I am confused --- you would like the vendor to stand up (in a NANOG presentation) and say: "Here is our widget, look how shiny it is.. Our device is better than $COMPETITOR because we do X, Y, Z, etc. We use the following heuristics <cough> and other vendors don't </ cough>"? To me this sound WAY more like a sales ploy (and, some of the other talks were much closer to this....).
Because, the "best" attack mitigation appliance vendor would seem to have 100% of their market, and thus, charge exhorbant prices for their product(s). When I brought this up with Mr. Vendor, his first reaction was to point out that the cost was less than a home-grown solution.
Yup... Said vendor does have a large market share -- by explaining how they collect the information they would have had to explain just how much of the Internet they instrument, which to me would have felt very salesey...
When I raised the question of open source software to do the same thing, his reaction was to ask: "oh? who's going to write it?" And that right there would seem to be a bit of bravado, perhaps fueled by a misunderstanding of the role that FOSS has played on the Net.
Yes, you can build your own attack mitigation solution (either based on OSS and / or from scratch), but there are limitations. Just saying "use OSS" doesn't make a fully formed solution spring into being, there are *large* investments needed in terms of time, effort, resource, scaling, training, lack of support, etc. While you *can* build a router using just OSS tools[0] there is a reason that most don't...
Fortunately -- and again, I am grateful for this -- the ISC was represented in the security BOF, presenting the SIE concept...as well as what applications _already exist_ to detect and mitigate various attacks. One demonstration that blew me away: detecting a botnet being set up for a phishing attack...and preventing the attack before it even started.
Great, I'm glad you liked that...
So in conclusion, I'll say this: the last NANOG I attended was NANOG 9 -- and i remember that being a more challenging environment for vendors. Probably the biggest problem discussed back then was head-of-line blocking on a vendor's switches. _That_ is the kind of content that i have found valuable, both on this list, and at a conference.
Hmmm, I remember some of these -- and I remember the "Our box does this way better than $OTHER_VENDOR" spin that was always put on this...
And so: If I weren't so knock-kneed in public venues, I would probably be doing what i would like to call on conference participants to do: if someone gives a presentation that includes their own proprietary black-box "solution", I think the best benefit for NANOG would be to point out alternatives.
Next time, please try and overcome your fear (although, I will happily point out that I haven't -- even saying "sorry, only time for 1 more question" gives me sweaty palms, makes me feel queasy, etc. What helps is to remember just how badly most of the other people here speak and that no-one cares) -- other (sane and realistic) solutions are always welcomed...
-Scott p.s. sorry for the long post.
W [0]: OMG, have I just kicked off the "Liinux / BSD as your core router" discussion again?!