On Thu, 21 September 2006 21:01:51 -0400, Richard A Steenbergen wrote: [..]

Seriously Gadi, what *possible* relevence could this have to network operations?

that, and a thread where half of the posts are from the initial poster himself anyway. but then, happily watching him, at least he is creative in topics... i am mentally killfilling his threads anyway, less and less relevant. it is scary what stuff is discussed lately. -ako

Well said. He can't respond right now, his computer has been infected. -----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of Michael.Dillon@btradianz.com Sent: Friday, September 22, 2006 5:18 AM To: nanog@merit.edu Subject: Have you really got clue?

that, and a thread where half of the posts are from the initial poster himself anyway. but then, happily watching him, at least he is creative in topics... i am mentally killfilling his threads anyway, less and less relevant. it is scary what stuff is discussed lately.

-ako

OK, Alexander Koch. You apparently have clue and you apparently know what *IS* on topic for this mailing list. Instead of posting an off-topic message like the one above, kindly post a message listing *ALL* of the topics that belong on this list. And if anyone else here thinks they know what is on topic, please tell us. I am getting bored by the flood of negative messages that say only "You can't say that here". Please stop telling us what you cannot say on NANOG. If you really must register your discontent with a message, then at least take the time to list some of the topics that belong on the list. What is NANOG all about? What is relevant to network operations? Is NANOG a narrowly focused technical list for a small group of technical specialists? Or is it some kind of broader industry-focused list that covers many issues relevant to the industry? --Michael Dillon

On 9/22/06, Laurence F. Sheldon, Jr. <LarrySheldon@cox.net> wrote:

It is pretty simple, really. These are examples of the topics that are on-topic.

1. "that posting is off-topic". 2. "somebody with clue from ${SmallUnknownOperator} (e.g. AOL) please contact me off list about a connectivity issue.:

Now that we're firmly into offtopic territory - http://www.kitenet.net/~joey/blog/entry/thread_patterns.html

Here's how to subscribe to mailing lists with a combined total posts of 2000 or more per day, and live. It's all about pattern recognition.

[snip] -- Suresh Ramasubramanian (ops.lists@gmail.com)

On Thu, 21 Sep 2006, Bill Sehmel wrote:

Gadi Evron wrote:

...

Hi guys, several ISP's are experiencing a flood of calls from customers who get failed installations of the recent IE 0day - VML - (vgx.dll).

If you are getting such floods too, this is why.

This is currently discussed on the botnets@ list, as raised by Cox, and I figured I will float it out here.

No patch is currently available from Microsoft, workaround are available.

Gadi.

And this has to do with Network Operations in what way?

In my book, if very large ISPs abuse desks become saturated, this is a problem ISPs face. Most ISPs would like to know how to respond to these questions, as well as know what's going on. Are you telling me tech support overflow at this immense scale does not affect the ISP and its network staff as well? It's not BGP, it's on-topic to others here. Gadi.

On Fri, 22 Sep 2006, Christopher L. Morrow wrote:

On Thu, 21 Sep 2006, Gadi Evron wrote:

...

Are you telling me tech support overflow at this immense scale does not affect the ISP and its network staff as well?

define 'immense scale' ... no calls here... so 'immense scale' in this case is 'nothing'.

No, one thing you might say is that increased (channelling Vijay here...) calls from customers means increased 'Support Cost' and decreased profit

Thank you for providing me with a correct explanation.

margin over time. I'd also say: 1) how is this different from a large scale network outage for a provider

Exactly the same, only seen at a few, so is likely to be seen with others.

2) how is this different from any other large worm outbreak thing

It's not.

3) is this blackworm all over again? (all hype no bite... byte?)

A lot of bite. Unfortunately. Every month on the third many still lose their files. What was interesting to nanog then was the IMMENSE global cooperation and coordination, encompassing too many and working, to mitigate it. Unless some us us, others here try and keep nanog in the loop. I know this interested many here, and nanog is the best way to reach them. Such occasional operational issues not interesting to you are interesting to us. These emails cause more disturbance. Is nanog to be BGP only? Please let me know and I won't email these here. Simple enough. If not, we all take note of what is interesting to us. Gadi.

-Chris

On Thu, 21 Sep 2006, Dave Stewart wrote:

At 10:28 PM 9/21/2006, you wrote:

...

...

2) how is this different from any other large worm outbreak thing

It's not.

Which makes it operational in which sense?

I'm starting to think that these "alerts" need to be filed along with the daily "OMG, evil people are taking over your computer if you don't send this to at least 10 people" IMs.

Paranoia has its place, but this ain't the place.

The report is NOT paranoia. Several LARGE user ISPs suffer immensely from this. Use this information if it is useful to you and you encounter the same problems. Thanks, Gadi.

On Fri, 22 Sep 2006, Adrian Chadd wrote:

On Thu, Sep 21, 2006, Gadi Evron wrote:

...

...

Paranoia has its place, but this ain't the place.

The report is NOT paranoia. Several LARGE user ISPs suffer immensely from this. Use this information if it is useful to you and you encounter the same problems.

Does it impact the network operation?

Eg, does it adversely affect the network? (say, like Beagle did.)

Not like Bagle did, to my knowledge. That said, this is spreading at an increasing rate that is unbelievable. That means worms, bots, and yes, ISP support, network and system personnel time depending on ISP.

Adrian

On Fri, Sep 22, 2006 at 12:01:58PM +0800, Adrian Chadd wrote:

On Thu, Sep 21, 2006, Gadi Evron wrote:

...

...

Paranoia has its place, but this ain't the place.

The report is NOT paranoia. Several LARGE user ISPs suffer immensely from this. Use this information if it is useful to you and you encounter the same problems.

Does it impact the network operation?

Eg, does it adversely affect the network? (say, like Beagle did.)

I was thinking sql-slammer, massive flood causing signifcant amount of network infrastructure to go down. (people on low speed links with large blocks of address space were DoS'ed off the network). I don't think of drive-by browser/desktop infection as a networking issue, more of an end-host issue. - Jared -- Jared Mauch | pgp key available via finger from jared@puck.nether.net clue++; | http://puck.nether.net/~jared/ My statements are only mine.

jared@puck.nether.net (Jared Mauch) writes:

I was thinking sql-slammer, massive flood causing signifcant amount of network infrastructure to go down. (people on low speed links with large blocks of address space were DoS'ed off the network).

right.

I don't think of drive-by browser/desktop infection as a networking issue, more of an end-host issue.

given that "network operations" now includes all kinds of non-bgp activities like datacenter design, tcp syn flood protection, nonrandom initial tcp sequence number prediction, and a googolplex or two of other issues, i've assumed that the hardcore bgp engineering community now meets elsewhere. (i wouldn't be needed or welcome "there" if so, so i'm just guessing.) so, for lack of a better forum, "things that can beat the hell out of your abuse desk" does indeed seem like safe fare for nanog@ in 2006, even though in 1996 maybe not so much so. (hell, in 1996 one could still send MIME attachments to abuse desks, since they were generally running solaris on NCD terminals rather than microsoft outlook, and attachments were "just opaque data", grrr.) can we all agree to stop shooting the messenger? every time gadi speaks up here, three or four folks bawl him out for being off-topic. time has proved that (a) gadi's not going to STFU no matter whether he's flamed or isn't, (b) those flaming arrows sticking out of his chest don't seem to injure him at all, (c) the flames completely outweigh gadi's own original posts, and (d) some of the folks lurking here actually tell me that they benefit from gadi's stuff. henceforth if you see a post, a poster, or a thread that you aren't interested in, "just hit delete". it'll save more bandwidth than flaming about it would. -- ISC Training! October 16-20, 2006, in the San Francisco Bay Area, covering topics from DNS to DHCP. Email training@isc.org. -- Paul Vixie

On Fri, 22 Sep 2006 10:11:20 +0100 Michael.Dillon@btradianz.com wrote:

Or perhaps BGP engineering hasn't changed in so many years that it is now more than adequately covered by books, certificate courses, and internal sharing of expertise. Lists are good for things that are new or confusing or difficult. BGP no longer fits into those categories.

In other words, this should be a focussed, low volume list.

...

and (d) some of the folks lurking here actually tell me that they benefit from gadi's stuff.

And, no doubt, they tell Gadi too which is why he continues to post on this list and does not seem to be wounded by the flaming arrows sent his way.

In other words, the some people think that the goal of a mailing list should be to keep a minimum volume of email going through it rather than keeping it focussed and useful. -- D'Arcy J.M. Cain <darcy@druid.net> | Democracy is three wolves http://www.druid.net/darcy/ | and a sheep voting on +1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.

To the people who say we throw in the towel and just say "Gadi will never stop posting off-topic crap, so why bother trying to correct him?", I'd suggest that this is a self-defeating attitude. Not only because Gadi could actually be posting useful stuff if set on the right path as to what is appropriate and what is not, but because 10,000 other people are going to be reading that post and thinking that this is appropriate subject matter. One off-topic post you can delete, but an entire list which has been co-opted by off-topic material can not be fixed.

I agree with you 100%. Please give us your list of *ALL* the topics that you think are appropriate for this list. --Michael Dillon P.S. Note that I do not agree that anyone has yet tried to "correct" Gadi. All I have seen is bellyaching on a personal level, i.e. person A does not like person B's message. To set everyone on the right path we need a description of the path itself.

On 22 Sep 2006, at 11:06, Lincoln Dale wrote:

...

P.S. Note that I do not agree that anyone has yet tried to "correct" Gadi.

i guess what i've found most bemusing about this whole thread is -- i went looking for the first email Gadi posted.

turns out that his posting habits have convinced Outlook that his email is junk - and _all_ of his posts are in the "Junk EMail" folder.

i was bemused. jury is out of Outlook is showing self-intelligence or not!

cheers,

lincoln.

Could we please close this thread now? I believe it is well off-topic. Thank you

Once again, ONE arguably off-topic post, followed by a non-stop stream of DOZENS of messages, for days, by self-appointed listcops. I'm sorry if the only thing which prompts you, and you know who you are, to post is that little rush of self-righteous adrenaline upon seeing a message you think is conceivably off-topic but resist the urge and sit on your hands or only send it to your imagined offender. It's a lot like shouting at the television set. Or, better, if you see something off-topic, POST A MESSAGE YOU FEEL IS ON-TOPIC, lead by example rather than by whining. Few things energize us more than another's sin. -- -Barry Shein The World | bzs@TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: 800-THE-WRLD | Login: Nationwide Software Tool & Die | Public Access Internet | SINCE 1989 *oo*

On Thu, 21 Sep 2006, Joel Jaeggli wrote:

Gadi, your initial query lacked the factual background that would have been useful for someone to decide if it was relevant to them or not. While I do believe that the intersection of host and applications issues and networking has applicability here I will make two observations that I hope are not wildly off the mark.

Many of the people on the operations side of networks do not spend a lot of time on security mailing lists. They also don't spend a lot of time looking into their own support organizations until until problems get escalated to them, so your initial post could have used more background.

Even in an enterprise it's really hard to justify the expenditure that a rapid response to a host security problem involves. For an isp which is not likely to be in the position to recover the cost of being reactive let alone pro-active I can't imagine how they would possibly support desktop issues like this.

Thank you, I will make sure and learn from this in the future! Gadi.

joelja

...

Thanks,

Gadi.

-- ------------------------------------------------------------------------ Joel Jaeggli Unix Consulting joelja@uoregon.edu GPG Key Fingerprint: 5C6E 0104 BAF0 40B0 5BD3 C38B F000 35AB B67F 56B2

Paul Vixie wrote:

joelja@uoregon.edu (Joel Jaeggli) writes:

...

Even in an enterprise it's really hard to justify the expenditure that a rapid response to a host security problem involves. For an isp which is not likely to be in the position to recover the cost of being reactive let alone pro-active I can't imagine how they would possibly support desktop issues like this.

<snip>

the perfect storm is a million extra customers calling over the course of a week just to explain that they have "exploding DLL symptoms" and listen to a "live operator" tell them that this isn't a network problem and they should contact the dealer where they bought their computer, which is likely CostCo. assuming that this takes less than 60 seconds per affected customer, it's still a nasty unbudgeted expense and as a secondary burn it will make real network problems harder to report.

Indeed. I'm fairly certain that in the life-cycle of some network maladies that decision has to be made as to whether you want to go out of business sooner (no more customers) or later (costs). When given the choice, I prefer the later. -- ------------------------------------------------------------------------ Joel Jaeggli Unix Consulting joelja@uoregon.edu GPG Key Fingerprint: 5C6E 0104 BAF0 40B0 5BD3 C38B F000 35AB B67F 56B2

sean@donelan.com (Sean Donelan) writes:

For assistance with Microsoft security issues in the US, call (866) PC-SAFETY

according to http://www.eweek.com/article2/0,1895,2019162,00.asp, microsoft has not released a patch for the VML thing, so calling (866) PC-SAFETY isn't going to be a universal fix (and who will $user call after that, we wonder?) according to http://www.websense.com/securitylabs/alerts/alert.php?AlertID=628, there is now malware-in-the-field that exploits the VML thing. and according to http://www.auscert.org.au/render.html?it=6771, there's already phishing. last but not least, according to http://isotf.org/zert/ there is a non-MSFT patch for the VML thing. i don't expect ISP's to recommend its use, due to liability reasons, but mentioning it or even proactively notifying about it might be a way to get people off the phone (or keep them from calling in). (i'll remove the ISC training ad from my .signature for this post, since i've gone way over my NANOG quota here -- three messages in 24 hours, oops.) -- Paul Vixie

On Sep 21, 2006, at 10:11 PM, Christopher L. Morrow wrote:

On Thu, 21 Sep 2006, Gadi Evron wrote:

...

Are you telling me tech support overflow at this immense scale does not affect the ISP and its network staff as well?

define 'immense scale' ... no calls here... so 'immense scale' in this case is 'nothing'.

I'm seeing email saying my employer's (large broadband) call centers are taking extremely high call volumes due they believe to this exploit. I don't think this is a case of crying wolf, since there are apparently several broadband providers who are getting hit with this, based on Gadi's email. I'll leave the flamewar as to whether this is on topic for NANOG or not to the experts. Bob