Microsoft's Black Tuesday bandwidth impact?
Every month I look at my upstream bandwidth graphs and I see no blip in the hours before 3 am on Microsoft's Black Tuesday. I would think that with the thousands of PCs out on our network downloading updates around that time that I would see *something*. I know every Black Tuesday I see my three PC's blinking a logon screen. Are MSFT's monthly updates really a non-event in regards to internet bandwidth? Frank
On Wed, Jan 09, 2008 at 02:47:53PM -0600, Frank Bulk wrote:
Are MSFT's monthly updates really a non-event in regards to internet bandwidth?
Most of these things are pushed out to some sort of CDN or multiple CDN networks. Compared to lots of other things, i'm sure the downloads are much smaller when compared to things like p2p, streaming media foo and legal video downloads. A lot of IT departments also use localized update servers so they get one copy for their 1000's of machines then do the distro internally to their network. Even at my house I run most everything through a caching proxy since compared to my "internet" speed, the disk+ethernet+squid provides a reasonable hitrate. I think there were 3 patches this time and they were quite small overall. - Jared -- Jared Mauch | pgp key available via finger from jared@puck.nether.net clue++; | http://puck.nether.net/~jared/ My statements are only mine.
Jared Mauch wrote:
On Wed, Jan 09, 2008 at 02:47:53PM -0600, Frank Bulk wrote:
Are MSFT's monthly updates really a non-event in regards to internet bandwidth?
Most of these things are pushed out to some sort of CDN or multiple CDN networks. Compared to lots of other things, i'm sure the downloads are much smaller when compared to things like p2p, streaming media foo and legal video downloads.
When I last had something to do at an ISP[1], we would regularly see our Akamai deployment jump from 10-20Mbps at 8am on Wednesday morning[2], to 40-60Mbps by 9am for an hour or two, around Patch Tuesday. We found it very noticeable then. I don't see it so noticeable on a carrier network doing ~8Gbps with hundreds of K of subscribers, these days. aj [1] 2004 [2] NZ is some hours ahead of the US timed patch releases. 8am-9am is when most people turned their PCs on in the morning, which then appeared to commence downloading.
On Wed, 09 Jan 2008 14:47:53 CST, Frank Bulk said:
Are MSFT's monthly updates really a non-event in regards to internet bandwidth?
How far is it to the nearest Akamai server from your network? I know there's one across the hall, so lot of local stuff pounds that rather than our off-site links.
Our upstream provider has one, so in that case, we would still see it on our pipe. It appears that it's not a blip on anyone else's radar, so I'll stop looking for it. =) Frank -----Original Message----- From: Valdis.Kletnieks@vt.edu [mailto:Valdis.Kletnieks@vt.edu] Sent: Wednesday, January 09, 2008 3:08 PM To: frnkblk@iname.com Cc: nanog@nanog.org Subject: Re: Microsoft's Black Tuesday bandwidth impact? On Wed, 09 Jan 2008 14:47:53 CST, Frank Bulk said:
Are MSFT's monthly updates really a non-event in regards to internet bandwidth?
How far is it to the nearest Akamai server from your network? I know there's one across the hall, so lot of local stuff pounds that rather than our off-site links.
I think you misunderstand how Automatic Updates work. The "3 AM" default setting simply defines when Windows will install updates. The actual detection and download of updates (including the file that the scan engine uses) are not done at 3 AM. Using default settings, Windows will run an update check every 22-17.6 hours. The 20% variance is to help prevent floods from computers booted up at the same (for example, a computer lab). This process will: check to see if there is a new update detection file, download an updated list, run a scan to see if the computer needs updates, and download needed updates. Microsoft releases updates on Tuesday at 10:00 AM PST (though some months they have been a few hours late). So to monitor the impact of the updates, look at the traffic between Tuesday 10:00 AM PST through Wednesday 8:00 AM PST. Note, not all computers will get their updates at this time. The most simplest example are computers that are off. But also, many people disable/modify their Automatic Updates settings, while others will use the Windows Update site. Hope that helps, Adam Stasiniewicz -----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of Frank Bulk Sent: Wednesday, January 09, 2008 2:48 PM To: nanog@nanog.org Subject: Microsoft's Black Tuesday bandwidth impact? Every month I look at my upstream bandwidth graphs and I see no blip in the hours before 3 am on Microsoft's Black Tuesday. I would think that with the thousands of PCs out on our network downloading updates around that time that I would see *something*. I know every Black Tuesday I see my three PC's blinking a logon screen. Are MSFT's monthly updates really a non-event in regards to internet bandwidth? Frank
Adam: Thanks for that detailed explanation. I was under the assumption the MSFT released updates just hours before 3 am (which is also a staggered target across the time zones). With almost a day of lead time (plus computers turned off, Automatic Updates turned off, local SUS servers), the network impact would be minimal, and with day-to-day traffic as varied as it is, probably imperceptible if I compared it to Tuesdays the week before or after. Frank -----Original Message----- From: Stasiniewicz, Adam [mailto:stasinia@msoe.edu] Sent: Wednesday, January 09, 2008 3:22 PM To: frnkblk@iname.com; nanog@nanog.org Subject: RE: Microsoft's Black Tuesday bandwidth impact? I think you misunderstand how Automatic Updates work. The "3 AM" default setting simply defines when Windows will install updates. The actual detection and download of updates (including the file that the scan engine uses) are not done at 3 AM. Using default settings, Windows will run an update check every 22-17.6 hours. The 20% variance is to help prevent floods from computers booted up at the same (for example, a computer lab). This process will: check to see if there is a new update detection file, download an updated list, run a scan to see if the computer needs updates, and download needed updates. Microsoft releases updates on Tuesday at 10:00 AM PST (though some months they have been a few hours late). So to monitor the impact of the updates, look at the traffic between Tuesday 10:00 AM PST through Wednesday 8:00 AM PST. Note, not all computers will get their updates at this time. The most simplest example are computers that are off. But also, many people disable/modify their Automatic Updates settings, while others will use the Windows Update site. Hope that helps, Adam Stasiniewicz -----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of Frank Bulk Sent: Wednesday, January 09, 2008 2:48 PM To: nanog@nanog.org Subject: Microsoft's Black Tuesday bandwidth impact? Every month I look at my upstream bandwidth graphs and I see no blip in the hours before 3 am on Microsoft's Black Tuesday. I would think that with the thousands of PCs out on our network downloading updates around that time that I would see *something*. I know every Black Tuesday I see my three PC's blinking a logon screen. Are MSFT's monthly updates really a non-event in regards to internet bandwidth? Frank
On Wed, 9 Jan 2008, Stasiniewicz, Adam wrote:
I think you misunderstand how Automatic Updates work. The "3 AM" default setting simply defines when Windows will install updates. The actual detection and download of updates (including the file that the scan engine uses) are not done at 3 AM.
Using default settings, Windows will run an update check every 22-17.6 hours. The 20% variance is to help prevent floods from computers booted up at the same (for example, a computer lab). This process will: check to see if there is a new update detection file, download an updated list, run a scan to see if the computer needs updates, and download needed updates.
On a similar note, anti virus companies add a delta of rand out of 300 minutes or so, for the same reason--coupled with the need of the net connection to not be self-DDoS'd. I find online game updates to be much more interested, bandwidth-wise, but never looked at MS's update as in organizations I was with it was controlled by a local centralized server (or 100).
Microsoft releases updates on Tuesday at 10:00 AM PST (though some months they have been a few hours late). So to monitor the impact of the updates, look at the traffic between Tuesday 10:00 AM PST through Wednesday 8:00 AM PST. Note, not all computers will get their updates at this time. The most simplest example are computers that are off. But also, many people disable/modify their Automatic Updates settings, while others will use the Windows Update site.
Hope that helps, Adam Stasiniewicz
-----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of Frank Bulk Sent: Wednesday, January 09, 2008 2:48 PM To: nanog@nanog.org Subject: Microsoft's Black Tuesday bandwidth impact?
Every month I look at my upstream bandwidth graphs and I see no blip in the hours before 3 am on Microsoft's Black Tuesday. I would think that with the thousands of PCs out on our network downloading updates around that time that I would see *something*. I know every Black Tuesday I see my three PC's blinking a logon screen.
Are MSFT's monthly updates really a non-event in regards to internet bandwidth?
Frank
On Thu, 10 Jan 2008, Gadi Evron wrote:
On a similar note, anti virus companies add a delta of rand out of 300 minutes or so, for the same reason--coupled with the need of the net connection to not be self-DDoS'd.
I find online game updates to be much more interested, bandwidth-wise, but never looked at MS's update as in organizations I was with it was controlled by a local centralized server (or 100).
Game patches are a different animal, the bandwidth profile is very different, since it has a heavy user demand on it, with patch time determined by the user and is often as close to release as possible. Very often, the patch is a content update that'll include graphic content. Blizzard did the right thing going with a torrent style patcher, even if they use a CDN to seed it. In my experience, they're one of the few that does/did that. - billn
On Wed, 9 Jan 2008, Frank Bulk wrote:
Every month I look at my upstream bandwidth graphs and I see no blip in the hours before 3 am on Microsoft's Black Tuesday. I would think that with the thousands of PCs out on our network downloading updates around that time that I would see *something*. I know every Black Tuesday I see my three PC's blinking a logon screen.
Are MSFT's monthly updates really a non-event in regards to internet bandwidth?
Users are too far from the firehose to feel the more interesting effects. That said, it's hit or miss, from month to month. If you have peering to a CDN network (llnw, akam, etc), you'll certainly see Patch Day roll through, since you're sitting on the aggregation of a large flow of data. As an end user, especially in an enterprise with admin's that are worth anything, you're not talking about a massive amount of data, in many cases. Service packs, sure, those are generally a bit bigger, but hotfixes and the like, usually pretty small. I don't even notice patches on my home connection, since they're a drop in the bucket compared to all the other content rolling around. Youtube and similar content flows are more noticeable. I think the only enterprise users who would notice a large influx of data are the ones who don't run caches. - billn
I actually speak for an ISP, not an enterprise at this time -- my apologies for not making it more clear. When I said "our network" I was really referring to our residential and business broadband subscribers. Among our business subscribers, only a handful actually have SUS in place. Frank -----Original Message----- From: Bill Nash [mailto:billn@billn.net] Sent: Wednesday, January 09, 2008 3:36 PM To: Frank Bulk Cc: nanog@nanog.org Subject: Re: Microsoft's Black Tuesday bandwidth impact? On Wed, 9 Jan 2008, Frank Bulk wrote:
Every month I look at my upstream bandwidth graphs and I see no blip in
hours before 3 am on Microsoft's Black Tuesday. I would think that with
the the
thousands of PCs out on our network downloading updates around that time that I would see *something*. I know every Black Tuesday I see my three PC's blinking a logon screen.
Are MSFT's monthly updates really a non-event in regards to internet bandwidth?
Users are too far from the firehose to feel the more interesting effects. That said, it's hit or miss, from month to month. If you have peering to a CDN network (llnw, akam, etc), you'll certainly see Patch Day roll through, since you're sitting on the aggregation of a large flow of data. As an end user, especially in an enterprise with admin's that are worth anything, you're not talking about a massive amount of data, in many cases. Service packs, sure, those are generally a bit bigger, but hotfixes and the like, usually pretty small. I don't even notice patches on my home connection, since they're a drop in the bucket compared to all the other content rolling around. Youtube and similar content flows are more noticeable. I think the only enterprise users who would notice a large influx of data are the ones who don't run caches. - billn
On Wed, 9 Jan 2008, Frank Bulk wrote:
I actually speak for an ISP, not an enterprise at this time -- my apologies for not making it more clear. When I said "our network" I was really referring to our residential and business broadband subscribers. Among our business subscribers, only a handful actually have SUS in place.
Even so, as mentioned in another piece of the thread, in combination with the timing spread for the update download, and the comparitive size of some of the updates compared to other content, I think you'll notice Xbox/Wii updates, Ron Paul spam, and Bittorrent releases more than you'll notice MSFT patches. - billn
Frank
-----Original Message----- From: Bill Nash [mailto:billn@billn.net] Sent: Wednesday, January 09, 2008 3:36 PM To: Frank Bulk Cc: nanog@nanog.org Subject: Re: Microsoft's Black Tuesday bandwidth impact?
On Wed, 9 Jan 2008, Frank Bulk wrote:
Every month I look at my upstream bandwidth graphs and I see no blip in
hours before 3 am on Microsoft's Black Tuesday. I would think that with
the the
thousands of PCs out on our network downloading updates around that time that I would see *something*. I know every Black Tuesday I see my three PC's blinking a logon screen.
Are MSFT's monthly updates really a non-event in regards to internet bandwidth?
Users are too far from the firehose to feel the more interesting effects. That said, it's hit or miss, from month to month. If you have peering to a CDN network (llnw, akam, etc), you'll certainly see Patch Day roll through, since you're sitting on the aggregation of a large flow of data. As an end user, especially in an enterprise with admin's that are worth anything, you're not talking about a massive amount of data, in many cases. Service packs, sure, those are generally a bit bigger, but hotfixes and the like, usually pretty small. I don't even notice patches on my home connection, since they're a drop in the bucket compared to all the other content rolling around. Youtube and similar content flows are more noticeable.
I think the only enterprise users who would notice a large influx of data are the ones who don't run caches.
- billn
Frank Bulk wrote:
Every month I look at my upstream bandwidth graphs and I see no blip in the hours before 3 am on Microsoft's Black Tuesday. I would think that with the thousands of PCs out on our network downloading updates around that time that I would see *something*. I know every Black Tuesday I see my three PC's blinking a logon screen.
Are MSFT's monthly updates really a non-event in regards to internet bandwidth?
Do you have Akamai servers locally on-net? -- Jay Hennigan - CCIE #7880 - Network Engineering - jay@impulse.net Impulse Internet Service - http://www.impulse.net/ Your local telephone and internet company - 805 884-6323 - WB6RDV
I made a short presentation related this topic in NANOG38. http://www.nanog.org/mtg-0610/tarui.html Recently, we can see a decline during the maintenance at the major video streaming site in Japan. Our statistics: http://www.jpnap.net/snapshot/ Yukiyasu Tarui On Wed, 9 Jan 2008 14:47:53 -0600
Every month I look at my upstream bandwidth graphs and I see no blip in the hours before 3 am on Microsoft's Black Tuesday. I would think that with the thousands of PCs out on our network downloading updates around that time that I would see *something*. I know every Black Tuesday I see my three PC's blinking a logon screen.
Are MSFT's monthly updates really a non-event in regards to internet bandwidth?
Frank
Thanks. Slide 8 of your PDF shows that what an ISP would see in a P2P heavy environment is that after the automatic application of Windows Updates a drop in traffic should be seen because the P2P desktop applications don't automatically restart after their PC reboots. I guess that means that if I don't see a drop a traffic, my end-users are behaving themselves! Frank -----Original Message----- From: Yukiyasu Tarui [mailto:tarui@mfeed.ad.jp] Sent: Wednesday, January 09, 2008 10:25 PM To: frnkblk@iname.com Cc: nanog@merit.edu Subject: Re: Microsoft's Black Tuesday bandwidth impact? I made a short presentation related this topic in NANOG38. http://www.nanog.org/mtg-0610/tarui.html Recently, we can see a decline during the maintenance at the major video streaming site in Japan. Our statistics: http://www.jpnap.net/snapshot/ Yukiyasu Tarui On Wed, 9 Jan 2008 14:47:53 -0600
Every month I look at my upstream bandwidth graphs and I see no blip in
hours before 3 am on Microsoft's Black Tuesday. I would think that with
the the
thousands of PCs out on our network downloading updates around that time that I would see *something*. I know every Black Tuesday I see my three PC's blinking a logon screen.
Are MSFT's monthly updates really a non-event in regards to internet bandwidth?
Frank
participants (10)
-
Alastair Johnson -
Bill Nash -
Frank Bulk -
Frank Bulk - iNAME
-
Gadi Evron -
Jared Mauch -
Jay Hennigan -
Stasiniewicz, Adam -
Valdis.Kletnieks＠vt.edu -
Yukiyasu Tarui