On Thu, 1 Apr 2021, Jean St-Laurent via NANOG wrote:
What happened is that it would create a kind of internal DDoS and they would all timed out and give a weird error message. Something very useful like Error Code 0x8098808 Please call our support line at this phone number.
If only there was a way to address the Thundering Herd problem before the cloud. :)
This simple change to add 3 lines of code to add a random artificial boot penalty of few seconds, completely solve the problem.
Bingo. Now, the trick is to catch this before it causes an self-DDoS. This is a problem that has been recognised for decades and this is unfortunately a good example of how operational experience is still not being distributed properly. Too many managers think that operational work is obvious and just a result of common sense. It isn't. Cheers, Rob