I think you're leaving out a very viable possibility in your summary... What if BoA took a proactive approach and shut down their SQL environment (even though none of us known conclusively if they're a SQL or Oracle shop) to verify that it was in fact clean and not compromised. When you're talking about access to billions of dollars, it's not worth taking a chance. They might have actually followed proper security protocol and verified their systems were clean before re-activating them. Just a thought. -Dave

-----Original Message----- From: Alex Rubenstein [SMTP:alex@nac.net] Sent: Sunday, January 26, 2003 10:59 AM To: Ray Burkholder Cc: nanog@nanog.org Subject: RE: Banc of America Article

Let me summarize, then ask a question:

a) BoA uses the public internet for ATM transactions. The public internet was so dead, that every one of thier ATM machines was dead for many hours, even many hours longer than the public internet was dead.

b) BoA uses it's own network for it's on ATM transactions. Somewhere on the a public to private connection, a firewall wasn't doing it's job, or there wasn't a firewall. Things were broken for a while, until they were able to fix all thier SQL servers.

I guess my point is, if it were a), not every ATM would be dead all the time, and things would have been fixed in only a little while. Not many internet 'backbones' (at least ones BoA would have used for this application) were down as long as BoA's ATM's were.

On the other hand, I think it's more likely that BoA had unprotected SQL servers, and they got it. It took a long while for BoA IT people to make it out of bed saturday morning to fix the problem.

I still clearly say that I don't know what happened, and I did make assumptions (as I said in the original mail) -- but I'd still place my money on b).

On Sun, 26 Jan 2003, Ray Burkholder wrote:

...

Actually, I think too many assumptions were made.

Let's simplify.

We know UUNet traffic capabilities were reduced significantly. Uunet has many big customers. Other big carriers had similar affects on their networks, probably particularly at peering points.

We know many companies use public or private VPN services from major carriers such as these, and that both VPN types may use public internet carriers.

I think therefore that the only true conclusion we could say is that if BoA's traffic was not prioritized, it therefore suffered collateral damage primarily due to traffic not being able to get through between ATM's and the central processing center.

-- Alex Rubenstein, AR97, K2AHR, alex@nac.net, latency, Al Reuben -- -- Net Access Corporation, 800-NET-ME-36, http://www.nac.net --

IMPORTANT:The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.