Good Day, I have always been exposed to one vendor only so i can never compare but I am curious to know what every one here have seen in their lives on the below: 1) Which vendor has more bugs than others, what are the top 3 2) Who is doing a better job fixing them 3) What do you consider is a good job in fixing these bugs : response from technical support, educated support engineers
Good Day, <Sorry, previous email sent by mistake> I have always been exposed to one vendor only so i can never compare but I am curious to know what every one here have seen in their lives on the below: 1) Which vendor has more bugs than others, what are the top 3 ? 2) Who is doing a better job fixing/handling these bugs overall 3) What do you consider is a good job in fixing/handling these bugs : A) Response from technical support B) Educated support engineers being able to respond to questions C) Taking less time to identify bugs D) Less time in fixing them E) Transparent communication on their issues F) Transparency from their teams allow us to plan better for our network G) etc.....please add more 4) Specially Huawei, are they doing a good job or its a mess? I would like to try to do some rating and ranking when it comes to bugs but i need to know what i have to be looking at? Regards, Kim
On Sun, 20 Feb 2011 18:05:44 +0200, Kasper Adel said: (Disclaimer - I've never filed a bug report with Cisco or Juniper, but I've spent 3 decades filing bugs with almost everybody else in the computer industry, it seems... Questions like the ones you asked are almost always pointless unless the asker and answerer are sharing a set of base assumptions. In other words, "which one is best/worst?" is a meaningless question unless you either tell us what *your* criteria are in detail, or are willing to listen to advice that uses other criteria (without stating how they're different from yours).
1) Which vendor has more bugs than others, what are the top 3
More actual bugs, more known and acknowledged bugs, or more serious bugs that actually affect day to day operations in a major manner? The total number of actual bugs for each vendor is probably unknownable, other than "there's at least one more in there". The vendor probably can produce a number representing how many bug reports they've accepted as valid. The vendor's number is guaranteed to be different than the customer's number - how divergent, *and why*, probably tells you a lot about the vendor and the customer base. The vendor may be difficult about accepting a bug report, or the customer base may be clueless about what the product is supposed to be doing and calling in a lot of non-bugs - almost every trouble ticket closed with RTFM status is one of these non-bugs. If there's a lot of non-bugs, it usually indicates a documentation/training issue, not an actual software quality issue. And of course, bug severity *has* to be considered. "Router falls over if somebody in Zimbabwe sends it a christmas-tree packet" is different than "the CLI insists on a ;; where a ; should suffice". You may be willing to tolerate or work around dozens or even hundreds of the latter (in fact, there's probably hundreds of such bugs in your current vendor that you don't know about simply because they don't trigger in your environment), but it only takes 2 or 3 of the former to render the box undeployable.
2) Who is doing a better job fixing them
Again, see the above discussion of severity. If a vendor is good about fixing the real show-stoppers in a matter of hours or days, but has a huge backlog of fixes for minor things, is that better or worse than a vendor that fixes half of both serious and minor things? In addition, the question of how fixes get deployed matters too. If a vendor is consistently good about finding a root cause, fixing it, and then saying "we'll ship the fix in the next dot-rev release", is that good or bad? Remember that if they ship a new, updated, more-fixed image every week, that means you get to re-qualify a new image every week....
2) Who is doing a better job fixing them
Again, see the above discussion of severity. If a vendor is good about fixing the real show-stoppers in a matter of hours or days, but has a huge backlog of fixes for minor things, is that better or worse than a vendor that fixes half of both serious and minor things?
In addition, the question of how fixes get deployed matters too. If a vendor is consistently good about finding a root cause, fixing it, and then saying "we'll ship the fix in the next dot-rev release", is that good or bad? Remember that if they ship a new, updated, more-fixed image every week, that means you get to re-qualify a new image every week....
That also changes over time. A vendor that might have been slow to fix bugs 5 years ago might be completely different today or a vendor that was particularly buggy 5 years ago might be rock solid today and the opposite is also true. Vendors can put out buggy stuff when they had been very stable. Use cases also vary widely and a portion of the software that might be very buggy for some feature set might be completely transparent to you because you don't use those features. The question is widely variable and very dependent on the deployment. Generally, the simpler you build your network, the less bugs you are going to run into. The more features you use, the more likely you are to run into them.
Thanks Valdis. On Sun, Feb 20, 2011 at 9:43 PM, <Valdis.Kletnieks@vt.edu> wrote:
On Sun, 20 Feb 2011 18:05:44 +0200, Kasper Adel said:
(Disclaimer - I've never filed a bug report with Cisco or Juniper, but I've spent 3 decades filing bugs with almost everybody else in the computer industry, it seems... Questions like the ones you asked are almost always pointless unless the asker and answerer are sharing a set of base assumptions. In other words, "which one is best/worst?" is a meaningless question unless you either tell us what *your* criteria are in detail, or are willing to listen to advice that uses other criteria (without stating how they're different from yours).
I tried to put details and criteria below and yes i am mainly interested in Juniper, Cisco, Alcatel and Huawei Routers and Switches, mostly High End Equipment and yes i am willing to listen to advice on criteria, why wouldnt I :) ?
1) Which vendor has more bugs than others, what are the top 3
More actual bugs, more known and acknowledged bugs, or more serious bugs that actually affect day to day operations in a major manner?
What i wanted to ask is from the field experience of experts on the alias if there is a clear winner on which vendor has throughout history shown more bugs impacting operation and interrupting traffic....poor written code or bad internal testing, can we have some sort of a general assumption here or that is not possible?
The total number of actual bugs for each vendor is probably unknownable, other than "there's at least one more in there". The vendor probably can produce a number representing how many bug reports they've accepted as valid. The vendor's number is guaranteed to be different than the customer's number - how divergent, *and why*, probably tells you a lot about the vendor and the customer base. The vendor may be difficult about accepting a bug report, or the customer base may be clueless about what the product is supposed to be doing and calling in a lot of non-bugs - almost every trouble ticket closed with RTFM status is one of these non-bugs. If there's a lot of non-bugs, it usually indicates a documentation/training issue, not an actual software quality issue.
And of course, bug severity *has* to be considered. "Router falls over if somebody in Zimbabwe sends it a christmas-tree packet" is different than "the CLI insists on a ;; where a ; should suffice". You may be willing to tolerate or work around dozens or even hundreds of the latter (in fact, there's probably hundreds of such bugs in your current vendor that you don't know about simply because they don't trigger in your environment), but it only takes 2 or 3 of the former to render the box undeployable.
2) Who is doing a better job fixing them
Again, see the above discussion of severity. If a vendor is good about fixing the real show-stoppers in a matter of hours or days, but has a huge backlog of fixes for minor things, is that better or worse than a vendor that fixes half of both serious and minor things?
In addition, the question of how fixes get deployed matters too. If a vendor is consistently good about finding a root cause, fixing it, and then saying "we'll ship the fix in the next dot-rev release", is that good or bad? Remember that if they ship a new, updated, more-fixed image every week, that means you get to re-qualify a new image every week....
What you have mentioned is operations headache, so one questions comes to mind here is what are issues a vendor will never be able to find in their internal testing, i mean are there issues that will definitely be discovered on the customer networks or we can assume that software needs to come out with less number of sev1/2 bugs because internal testing is not doing a good job? thanks
participants (3)
-
George Bonser -
Kasper Adel -
Valdis.Kletnieks＠vt.edu