Valdis.Kletnieks@vt.edu writes:
On Thu, 10 Jun 2004 12:23:42 PDT, Eric Rescorla said:
I'm not sure we disagree. All I was saying was that I don't think we have a good reason to believe that the average bug found independently by a white hat is already known to a black hat. Do you disagree?
Actually, yes.
Non-obvious bugs (ones with a non 100% chance of being spotted on careful examination) will often be found by both groups. Let's say we have a bug that has a 0.5% chance of being found at any given attempt to find it. Now take 100 white hats and 100 black hats - compute the likelyhood that at least 1 attempt in either group finds it (I figure it as some 39% (1 - (0.995^100)). For bonus points, extend a bit further, and make multiple series of attempts, and compute the probability that for any given pair of 100 attempts, exactly one finds it, or neither finds it, or both find it. And it turns out that for that 39% chance, 16% of the time both groups will find it, 36% of the time exactly one will find it, and 48% of the time *neither* will find it.
The problem with this a priori analysis is that it predicts an incredibly high probability that any given bug will be found by white hats. However, in practice, we know that bugs persist for years without being found, so we know that that probability as a function of time must actually be quite low. Otherwise, we wouldn't see the data we actually see, which is a more or less constant stream of bugs at a steady rate. On the other hand, if the probability that a given bug will be found is low [0], then the chance that when you find a bug it will also be found by someone else is correspondingly low. -Ekr [0] Note that this doesn't require that the chance of finding any particular bug upon inspection of the code be very low high, but merely that there not be very deep coverage of any particular code section.