Are we thinking its going to get worse?? Did anyone else see the intelsat spacecraft failure last week?? Sent using my GCI BlackBerry ----- Original Message ----- From: Valdis.Kletnieks@vt.edu <Valdis.Kletnieks@vt.edu> To: Michael Dillon <wavetossed@googlemail.com> Cc: Paul Vixie <vixie@isc.org>; Robert E. Seastrom <rs@seastrom.com>; nanog@merit.edu <nanog@merit.edu> Sent: Sun Apr 11 08:36:05 2010 Subject: Re: Solar Flux (was: Re: China prefix hijack) On Sun, 11 Apr 2010 16:58:40 BST, Michael Dillon said:
Would a Faraday cage be sufficient to protect against cosmic ray bit-flipping and how could you retrofit a Faraday cage onto a rack or two of gear?
Scientists build neutrino detectors in mines 8,000 feet underground because that much rock provides *partial* shielding against cosmic rays causing spurious detection events. Fortunately, the sun emits almost no cosmic rays. It does however spew a lot of less energetic particles that will cause single-bit upsets in electronic gear. Time to double-check that all your gear has ECC ram - the problem with the UltraSparc CPUs last time was that they had some cache chips built by IBM. IBM said "Use these chips in an ECC config", but Sun didn't. The ions hit, and the resulting bit-flips crashed the machines. Incidentally, Sun sued IBM over that, and the judge basically said "Well, IBM *told* you not to do that up front. Suit dismissed". One of the other big issues will be noise on satellite and microwave links screwing your S/N ratio. The one that scares me? Inducted currents on long runs of copper. You get a 200-300 mile 765Kva transmission line, and a solar flare hits, the Earth's magnetic field gets dented, so the field lines move relative to the stationary copper cable, and suddenly you have several thousand extra amps popping out one end of that cable. Ka-blam. The big danger there is that many substations are not designed for that - so it would basically *permanently* destroy that substation and they'd get to replace it. And of course, that's a several-weeks repair even if it's the only one - and in that sort of case, there will be *dozens* of step-down transformers blown up the same afternoon. How long can you run on diesel? ;)
Warren Bailey <wbailey@gci.com> writes:
Are we thinking its going to get worse??
i am. looking at some local passive dns data (generated from ISC SIE), we find the following single bit errors by anchoring some searches at the known names and addresses of the f-root server. DNS does not have its own crc or checksum, it relies on UDP and IP for error detection (in which it is weak or nonexistant). i think it's probably time to formalize the study of bit errors in DNS, and then see if we can make a sensible mapping between what we observe and what the solar flux is. because if they're related and we're going to have a strong solar cycle then everybody's insurance rates are about to go up. re: ;; matches for IP = 192.5.5.241 (resource records) m.root-servers.net. IN A 192.5.5.241 a.root-servers.net. IN A 192.5.5.241 b.root-servers.net. IN A 192.5.5.241 c.root-servers.net. IN A 192.5.5.241 d.root-servers.net. IN A 192.5.5.241 e.root-servers.net. IN A 192.5.5.241 f.boot-servers.net. IN A 192.5.5.241 f.ro\127t-servers.net. IN A 192.5.5.241 f.rood-servers.net. IN A 192.5.5.241 f.roop-servers.net. IN A 192.5.5.241 f.root-cervers.net. IN A 192.5.5.241 f.root-sarvers.net. IN A 192.5.5.241 f.root-sebvers.net. IN A 192.5.5.241 f.root-servebs.net. IN A 192.5.5.241 f.root-serverc.net. IN A 192.5.5.241 f.root-servers.nut. IN A 192.5.5.241 f.root-servers.ned. IN A 192.5.5.241 f.root-servers.neu. IN A 192.5.5.241 f.root-servers.~et. IN A 192.5.5.241 f.root-servers. IN A 192.5.5.241 f.root-servess.net. IN A 192.5.5.241 f.root-servurs.net. IN A 192.5.5.241 f.root-serwers.net. IN A 192.5.5.241 f.root-serfers.net. IN A 192.5.5.241 f.root-survers.net. IN A 192.5.5.241 f.root=servers.net. IN A 192.5.5.241 f.rcot-servers.net. IN A 192.5.5.241 f.rgot-servers.net. IN A 192.5.5.241 f.rkot-servers.net. IN A 192.5.5.241 f.r\127ot-servers.net. IN A 192.5.5.241 g.root-servers.net. IN A 192.5.5.241 h.root-servers.net. IN A 192.5.5.241 i.root-servers.net. IN A 192.5.5.241 j.root-servers.net. IN A 192.5.5.241 k.root-servers.net. IN A 192.5.5.241 l.root-servers.net. IN A 192.5.5.241 v.root-servers.net. IN A 192.5.5.241 ;; matches for IP = 2001:500:2f::f (resource records) f.roop-servers.net. IN AAAA 2001:500:2f::f f.root-cervers.net. IN AAAA 2001:500:2f::f f.root-sarvers.net. IN AAAA 2001:500:2f::f f.root-servebs.net. IN AAAA 2001:500:2f::f f.root-servers.ned. IN AAAA 2001:500:2f::f f.root-servers.neu. IN AAAA 2001:500:2f::f f.root-servers.~et. IN AAAA 2001:500:2f::f f.root-serwers.net. IN AAAA 2001:500:2f::f f.root-serfers.net. IN AAAA 2001:500:2f::f f.root=servers.net. IN AAAA 2001:500:2f::f f.rcot-servers.net. IN AAAA 2001:500:2f::f f.rgot-servers.net. IN AAAA 2001:500:2f::f f.rkot-servers.net. IN AAAA 2001:500:2f::f ;; matches for name = f.root-servers.net (RRsets) f.root-servers.net. IN A 128.63.2.53 f.root-servers.net. IN A 128.63.2.53 f.root-servers.net. IN A 192.5.5.241 f.root-servers.net. IN A 128.63.2.53 f.root-servers.net. IN A 192.112.36.4 f.root-servers.net. IN A 128.8.10.90 f.root-servers.net. IN A 128.8.10.90 f.root-servers.net. IN A 192.5.5.241 f.root-servers.net. IN A 128.8.10.90 f.root-servers.net. IN A 202.12.27.33 f.root-servers.net. IN A 192.58.128.30 f.root-servers.net. IN A 192.58.128.30 f.root-servers.net. IN A 198.41.0.4 f.root-servers.net. IN A 192.5.5.225 f.root-servers.net. IN A 192.5.5.253 f.root-servers.net. IN A 192.5.5.255 f.root-servers.net. IN A 192.5.5.240 f.root-servers.net. IN A 192.5.5.241 f.root-servers.net. IN A 192.58.128.30 f.root-servers.net. IN A 192.5.5.241 f.root-servers.net. IN A 192.58.128.30 f.root-servers.net. IN A 192.228.79.201 f.root-servers.net. IN A 192.5.5.241 f.root-servers.net. IN A 192.58.128.30 f.root-servers.net. IN A 198.41.0.4 f.root-servers.net. IN A 192.5.5.241 f.root-servers.net. IN A 192.5.5.241 f.root-servers.net. IN A 192.5.5.241 f.root-servers.net. IN A 192.33.4.12 f.root-servers.net. IN A 192.5.5.241 f.root-servers.net. IN A 192.33.4.12 f.root-servers.net. IN A 192.58.128.30 f.root-servers.net. IN A 192.5.5.241 f.root-servers.net. IN A 192.33.4.12 f.root-servers.net. IN A 193.0.14.129 f.root-servers.net. IN A 192.5.5.241 f.root-servers.net. IN A 192.33.4.12 f.root-servers.net. IN A 198.32.64.12 f.root-servers.net. IN A 192.5.5.241 f.root-servers.net. IN A 192.33.4.12 f.root-servers.net. IN A 202.12.27.33 f.root-servers.net. IN A 192.5.5.241 f.root-servers.net. IN A 192.36.148.17 f.root-servers.net. IN A 192.5.5.241 f.root-servers.net. IN A 192.112.36.4 f.root-servers.net. IN A 192.5.5.241 f.root-servers.net. IN A 192.203.230.10 f.root-servers.net. IN A 192.5.5.241 f.root-servers.net. IN A 192.228.79.201 f.root-servers.net. IN A 192.5.5.241 f.root-servers.net. IN A 192.228.79.201 f.root-servers.net. IN A 193.0.14.129 f.root-servers.net. IN A 192.5.5.241 f.root-servers.net. IN A 192.228.79.201 f.root-servers.net. IN A 198.32.64.12 f.root-servers.net. IN A 192.5.5.241 f.root-servers.net. IN A 192.228.79.201 f.root-servers.net. IN A 202.12.27.33 f.root-servers.net. IN A 192.5.5.241 f.root-servers.net. IN A 193.0.14.129 f.root-servers.net. IN A 192.5.5.241 f.root-servers.net. IN A 193.0.14.129 f.root-servers.net. IN A 198.41.0.4 f.root-servers.net. IN A 192.5.5.241 f.root-servers.net. IN A 198.32.64.12 f.root-servers.net. IN A 192.5.5.241 f.root-servers.net. IN A 198.32.64.12 f.root-servers.net. IN A 198.41.0.4 f.root-servers.net. IN A 192.5.5.241 f.root-servers.net. IN A 198.41.0.4 f.root-servers.net. IN A 192.5.5.241 f.root-servers.net. IN A 198.41.0.4 f.root-servers.net. IN A 202.12.27.33 f.root-servers.net. IN A 192.5.5.241 f.root-servers.net. IN A 202.12.27.33 f.root-servers.net. IN A 192.5.5.248 f.root-servers.net. IN A 192.5.5.249 f.root-servers.net. IN A 192.5.5.250 f.root-servers.net. IN A 192.5.5.251 f.root-servers.net. IN A 192.5.5.252 f.root-servers.net. IN A 192.5.21.241 f.root-servers.net. IN A 192.21.5.241 f.root-servers.net. IN A 192.33.4.12 f.root-servers.net. IN A 192.33.4.12 f.root-servers.net. IN A 192.36.148.17 f.root-servers.net. IN A 192.36.148.17 f.root-servers.net. IN A 192.228.79.201 f.root-servers.net. IN A 192.112.36.4 f.root-servers.net. IN A 192.112.36.4 f.root-servers.net. IN A 192.203.230.10 f.root-servers.net. IN A 192.203.230.10 f.root-servers.net. IN A 192.203.230.10 f.root-servers.net. IN A 198.32.64.12 f.root-servers.net. IN A 192.228.79.201 f.root-servers.net. IN A 192.228.79.201 f.root-servers.net. IN A 193.0.14.129 f.root-servers.net. IN A 192.228.79.201 f.root-servers.net. IN A 198.32.64.12 f.root-servers.net. IN A 192.228.79.201 f.root-servers.net. IN A 202.12.27.33 f.root-servers.net. IN A 193.0.14.129 f.root-servers.net. IN A 193.0.14.129 f.root-servers.net. IN A 198.41.0.4 f.root-servers.net. IN A 198.32.64.12 f.root-servers.net. IN A 198.32.64.12 f.root-servers.net. IN A 198.41.0.4 f.root-servers.net. IN A 198.41.0.4 f.root-servers.net. IN A 198.41.0.4 f.root-servers.net. IN A 202.12.27.33 f.root-servers.net. IN A 202.12.27.33 f.root-servers.net. IN 4097 \# 4 c0 05 05 f1 f.root-servers.net. IN AAAA 2001:500:2f::3a:801e f.root-servers.net. IN AAAA 2001:500:2f:: f.root-servers.net. IN AAAA 2001:500:2f::1:0 f.root-servers.net. IN AAAA 2001:500:2f::14:0 f.root-servers.net. IN AAAA 2001:510:2f::f f.root-servers.net. IN AAAA 2101:500:2f::f f.root-servers.net. IN AAAA 2109:500:2f::f f.root-servers.net. IN LOC \# 16 20 01 05 00 00 2f 00 00 00 00 00 00 00 00 00 0f -- Paul Vixie KI6YSY
On 04/11/2010 06:02 PM, Paul Vixie wrote:
Warren Bailey <wbailey@gci.com> writes:
Are we thinking its going to get worse??
i am. looking at some local passive dns data (generated from ISC SIE), we find the following single bit errors by anchoring some searches at the known names and addresses of the f-root server. DNS does not have its own crc or checksum, it relies on UDP and IP for error detection (in which it is weak or nonexistant). i think it's probably time to formalize the study of bit errors in DNS, and then see if we can make a sensible mapping between what we observe and what the solar flux is.
because if they're related and we're going to have a strong solar cycle then everybody's insurance rates are about to go up.
And to top it all off, how many picojoules are stored in a modern ram cell compared to the same during the last sunspot peak. There is a hidden cost to memory density growth here... -- Pete
-----Original Message----- From: Pete Carah [mailto:pete@altadena.net] Sent: Sunday, April 11, 2010 8:41 PM To: nanog@nanog.org Subject: Re: Solar Flux
And to top it all off, how many picojoules are stored in a modern ram cell compared to the same during the last sunspot peak. There is a hidden cost to memory density growth here...
-- Pete
This cycle is currently predicted to be quite tame by recent standards. So far NASA is predicting cycle 24 to be much weaker than 23 was: http://solarscience.msfc.nasa.gov/images/f107_predict.gif http://solarscience.msfc.nasa.gov/predict.shtml
"George Bonser" <gbonser@seven.com> writes:
-----Original Message----- From: Pete Carah [mailto:pete@altadena.net] Sent: Sunday, April 11, 2010 8:41 PM To: nanog@nanog.org Subject: Re: Solar Flux
And to top it all off, how many picojoules are stored in a modern ram cell compared to the same during the last sunspot peak. There is a hidden cost to memory density growth here...
-- Pete
This cycle is currently predicted to be quite tame by recent standards. So far NASA is predicting cycle 24 to be much weaker than 23 was:
That said, nobody got it right in terms of the depth or length of the current valley, and we are in the bear skins and flint knives era of predicting Sol's activity. Also, a weak cycle (integration over a period of 11 years) does not mean there won't be brief periods of utter insanity. -r
participants (5)
-
George Bonser
-
Paul Vixie
-
Pete Carah
-
Robert E. Seastrom
-
Warren Bailey