On Fri, May 25, 2001 at 12:44:28AM -0400, jlewis@lewis.org wrote:

Email them a URL with a username and password.

This requires that you: 1) Have a web site with sufficient space to store the file. 2) Have it accessible, i.e. not firewalled on port 80. 3) Set up the accounts and passwords. It also requires that they: 1) Run a web browser. 2) Have the username and password available when they attempt to get the file. Exactly how is that more efficient than just mailing them the file? I'd completely agree with you for, say, a mailing list; but we're talking about person A wants the file, person B has the file. Why should either one of them jump through hoops? More importantly, why do you give a shit how they choose to transfer files to each other?

On Fri, 25 May 2001, John Fraizer wrote:

On Thu, 24 May 2001, Roeland Meyer wrote:

...

...

From: Steve Sobol [mailto:sjsobol@NorthShoreTechnologies.net] Sent: Thursday, May 24, 2001 2:51 PM

Shawn McMahon wrote:

...

...

TCP rate-limiting on outbound traffic to *:25 would also be extremely effective, particularly on unclassified customer traffic, and without the heavy-handed nature of denying all dial-up traffic. Rate-limiting doesn't interfere with low-volume legitimate mail, but it really cramps spam.

It interferes heavily with transmission of large files via email, though, and this *IS* a valid use of service.

The transmission of large files is not a valid use of email.

Is too... I send large documents regularly, via email. I just sent a 125 page word doc, with about 20 embedded Visio drawings and a bunch of embedded Excel spreadsheets. It was huge. Most of the recipients are on dialups with Win98. How else do you expect me to get it to them ... FTP? Most of them are NOT computer jocks.

How about this Roeland:

You send them an email that says:

" Because email is NOT intended to be a file transfer protocol and beyond that fact that I know you're on a low-bandwidth dialup account, please find below a link to the document I said I would send to you. On a 56K dialup connection, this will most likely take about 10 minutes for you to download. There are two links. One will retrieve the document via FTP and one will retrieve the document via HTTP. The HTTP link will most likely provide a faster download under most circumstances. I provided both to afford you the opportunity to choose when and how you retrieve the document.

http://www.mhsc.com/~rmeyer/Document-I-Promised.zip

ftp://ftp.mhsc.com/pub/users/rmeyer/Document-I-Promised.zip "

Very simple concept. It not only uses the right tool for the job but, also affords them the opportunity to retrieve the document when it is CONVENIENT for them. If I were a dialup user and somone sent me some HUGE attachment like that, I would consider it very rude.

Note to all salesdroids: If you want to be sure that I will NEVER do business with you, send me an email attachment.

--- John Fraizer EnterZone, Inc

If you were a dial-up user, chances are you wouldn't be able to do that. A few simple reasons come to mind: first, you wouldn't have any or not enough disk space on your system account (limited by quota) to store the file. Second, an average user probably wouldn't have the skill. Third, a .zip file will usually display as funny characters on a web browser - that's why ftp is needed. Fourth, you probably wouldn't have shell access and ftp space from your provider with a regular account. Fifth, assuming you would have all the toys, you would have to spend yourself the time to first upload the file, so that another may retrieve it. Sixth, if your file was a sensitive document, others would have public access to it, etc. So what's a regular user to do? Email it! Hence the legitimate use of email for transmission of large files. Most ISPs know that if they start limiting this privilege, users will migrate to someone that allows it. --Mitch NetSide

On Fri, May 25, 2001 at 07:02:35AM -0400, Jim Mercer wrote:

i regularly configure ISP's with a limit on the size of email messages. (generally 10meg, although i think 100k is probably better).

Generally 10MB. Now why is that, I wonder? Do you think people will be typing 10MB of text? Or even 100K of text? Why is it that the limit has to be that large, Jim? I can tell you from personal experience that accidentally setting it for 1MB instead of 10 will get you paged out of bed to "fix" it. Why is that, if only Roeland and I send large files in email?

...

Yes, BITFTP is currently restricted to users sending requests from sites on BITNET/EARN/NetNorth. And, yes, this is the direct result of your complaints.

Ah, so you're the one that fucked it up for everybody else, and obsoleted the instructions in so many O'Reilly books. :-)

On Fri, May 25, 2001 at 08:16:03AM -0400, Shawn McMahon wrote:

On Fri, May 25, 2001 at 07:02:35AM -0400, Jim Mercer wrote:

...

i regularly configure ISP's with a limit on the size of email messages. (generally 10meg, although i think 100k is probably better).

Generally 10MB. Now why is that, I wonder? Do you think people will be typing 10MB of text?

Or even 100K of text?

i accept, with the adoption of MIME standards, that people will use email to move the odd thing around. also, with some of the stupid message encodings these days, a simple "hello there" with background wallpaper and animated letterhead can easily jump over 100k.

Why is it that the limit has to be that large, Jim?

10meg was an arbitrary decision, and my personal preference. i'm not attempting to dictate policy here, i know that is futile. but, my hope is to inject some sound reasoning behind my decisions, such that others can benefit from my experience, or not, if they so choose.

I can tell you from personal experience that accidentally setting it for 1MB instead of 10 will get you paged out of bed to "fix" it.

Why is that, if only Roeland and I send large files in email?

i do get paged, less frequently since going from 5meg to 10meg, but my reaction to the client is the same. WTF are they sending that is that huge? i then explain to the client the possible reprecussions to their other users if they do not enforce some type of limit. boneheaded users who feel the need to send 400meg zipfiles to their buddies yahoo account. maybe the MTA i'm using (smail) is not appropriate. i don't think this is the issue, as in "normal" operation, the queues flush within reasonable expectations.

Ah, so you're the one that fucked it up for everybody else, and obsoleted the instructions in so many O'Reilly books. :-)

yep, that's me. and damn proud of it. you should really take the BITFTP issue in context though. at that time, much (very much) of the email and news moving around the "net" was being done using uucp over modems. our site (lsuc.uucp/lsuc.on.ca) was a voluntary, un-paid hub for hundreds of academic, personal, commercial, etc uucp sites in toronto. (directly 80+, and what ever nodes lived behind them). our site operated using an AT&T 3b2 with ESDI disks. this was reasonably state of the art given our budget. we also used a couple telebit trailblazers and a bank of 1200 baud modems. we were not commercial, and if someone wanted to step up and take over the responsibility, we were open to it. but, alas, we were it. and we were getting swamped. and i did something about it. -- [ Jim Mercer jim@reptiles.org +1 416 410-5633 ] [ Now with more and longer words for your reading enjoyment. ]

Sorry, can't resist replying here. In my limited (7 years) experience, 99% of all large file transfers via e-mail consist of dancing babies, horny snowmen, clumsy reindeer/monkeys/people movie clips. Oh, did I mention the plethora of cutesy jack-o-lanterns around October 31st? I also find it remarkable that no one seems to protect these 'sensitive' documents with PGP or another encryption method since we all know that e-mail is in plain text. What were you saying about ftp being insecure? David Leonard ShaysNet On Fri, 25 May 2001, Mitch Halmu wrote:

So what's a regular user to do? Email it! Hence the legitimate use of email for transmission of large files. Most ISPs know that if they start limiting this privilege, users will migrate to someone that allows it.

On Fri, 25 May 2001 11:45:13 EDT, Robert Blayzor said:

Right, but let's not leave out the attchment honorable mentions, things like ILOVEYOU, Snowwhite, etc. ;-) Perfect example of how email attachments can start a wildfire.

It's not the attachments that's the problem. It's companies that sell software that looks at attachments with a fundementally screwed-up security model. -- Valdis Kletnieks Operating Systems Analyst Virginia Tech

Actually, in a business environment (at least at my former company) the mailings were large power point presentations and visio docs and such.. All perfectly legitimate traffic. The funny thing about this is that it was usually to folks in the same office who had access to (and used) a central server. It would have been just as easy to send out a pointer to the location of the doc and less destructive. Is there anyone here who has NOT had to go in a rescue a mailbox that was too large for the client to load? (And yes, this happened with pine, outlook express, and a couple of others.. not just a random here or there.) And dare I mention problems with POP servers and timeouts over low speed links? On Fri, May 25, 2001 at 11:45:13AM -0400, Robert Blayzor wrote:

...

Sorry, can't resist replying here. In my limited (7 years) experience, 99% of all large file transfers via e-mail consist of dancing babies, horny snowmen, clumsy reindeer/monkeys/people movie clips. Oh, did I mention the plethora of cutesy jack-o-lanterns around October 31st? I also find it remarkable that no one seems to protect these 'sensitive' documents with PGP or another encryption method since we all know that e-mail is in plain text. What were you saying about ftp being insecure?

Right, but let's not leave out the attchment honorable mentions, things like ILOVEYOU, Snowwhite, etc. ;-) Perfect example of how email attachments can start a wildfire.

-- Robert Blayzor IP Network Engineer, BOFH BiznessOnline.com, Inc. rblayzor@thebiz.net noc@thebiz.net http://www.thebiz.net/

FreeBSD, Putting the 'Operating' back into OS! -- http://www.freebsd.org/

Alexei Roudnev wrote:

Few things are obvious:

(1) sender must be able to use simple _attach file_ feature to send a file, without extra skills; (2) receiver should be able to receive this file and open it as part of letter; without extra skills.

If you want to create additional service, you can (by request) retract attachments on the server, place them on the web (you can virus - scan them if you want and add virus warning) and include html reference into the mail itself;

It may be worth noting that AOL's mail service (which is only accessible via the proprietary AOL client or web pages) does just this. Attachments are kept on the server until the user explicitly chooses to download them.

but it cause a lot of problems if customer uses something other that openlook - express. For some cases it looks reasonable; for opthers, IMAP can solve the problem. And (of course) you need _smart_ anty-spam filters which can be turn on / off by the customer (and can have ON default).

Yes. Any system where messages are not kept on the server will have a problem with this. For instance, how will the server know when to delete the web-linked attachments? If the message is left on the server, this is a no-brainer - remove it when the message is deleted. But if it's on a system where people normally remove messages from the server prior to reading (like POP), then there is no good way to know when it is safe to delete the attachments from the web server.

My friends filter out SPAM by finding the same messages send to the hundred of the custoimers at onse - this simple method drop the SPAM percent dramatically (just again, it's important to allow receiving spam - it can be something except SPAM, so they provide some way to look through removed messages, as I know). Anyway, SPAm p[roblem and PARSEL problem are different ones.

The only problem with this is that if hundreds of customers are all subscribed to a single legitimate opt-in mailing list, this kind of filter will incorrectly tag it as spam. Depending on the size and makeup of your user base, this may or may not be a likely scenario. -- David

If you were a dial-up user, chances are you wouldn't be able to do that. A few simple reasons come to mind: first, you wouldn't have any or not enough disk space on your system account (limited by quota) to store the file. Second, an average user probably wouldn't have the skill. Third, a .zip file will usually display as funny characters on a web browser - that's why ftp is needed. Fourth, you probably wouldn't have shell access and ftp space from your provider with a regular account. Fifth, assuming you would have all the toys, you would have to spend yourself the time to first upload the file, so that another may retrieve it. Sixth, if your file was a sensitive document, others would have public access to it, etc.

Ignorance usually doesn't necessarily give someone the right to abuse services, regardless if they know they are doing it or not. Large email attachments are abusive to mail servers. Especially for the people that feel the need they need to CC it to ten different people. Large email attachments tax servers and delay other more important email from being delivered. SMTP is for mail, FTP/HTTP should be used for files. Not to mention 99% of the time they are very rude to receive. Just imagine, you're on vacation somewhere dialed in from a hotel that is charging you a buck a minute. You go to pick up the email that is really important to you, but you can't because some jerk decided that they would send you the most recent funny AVI they saw on the web, when they could have just sent you the URL. There are simple mechanisms to give users options to setup files via HTTP very easily. (or even FTP ie: ftp://user:pass@members.myisp.com) Browsers easily allow users to simply type in ONE URL (that can be bookmarked) and basically drag and drop their files right into a window. Very simple.

So what's a regular user to do? Email it! Hence the legitimate use of email for transmission of large files. Most ISPs know that if they start limiting this privilege, users will migrate to someone that allows it.

That doesn't fix anything. Many ISP's already restrict the size of incoming message. I know several that won't even allow anything larger than 1MB to be received. Also, most ISP's also offer a generate amount of free web space that comes with their dial-up account. (usually in the 10-20mb range) -- Robert Blayzor IP Network Engineer, BOFH BiznessOnline.com, Inc. rblayzor@thebiz.net noc@thebiz.net http://www.thebiz.net/ FreeBSD, Putting the 'Operating' back into OS! -- http://www.freebsd.org/

On Fri, 25 May 2001, Mitch Halmu wrote: [snip]

file. Second, an average user probably wouldn't have the skill. Third, a .zip file will usually display as funny characters on a web browser - that's why ftp is needed. Fourth, you probably wouldn't have shell access [snip]

Web browsers will only display 'funny characters' for .zip files if they don't have their mime-types sufficently configured. -- The comments and opinions expressed herein are those of the author of this message and may not reflect the policies of the Martin County Board of County Commissioners.

On Fri, 25 May 2001, Mitch Halmu wrote:

...

http://www.mhsc.com/~rmeyer/Document-I-Promised.zip

ftp://ftp.mhsc.com/pub/users/rmeyer/Document-I-Promised.zip "

Very simple concept. It not only uses the right tool for the job but, also affords them the opportunity to retrieve the document when it is CONVENIENT for them. If I were a dialup user and somone sent me some HUGE attachment like that, I would consider it very rude.

Note to all salesdroids: If you want to be sure that I will NEVER do business with you, send me an email attachment.

--- John Fraizer EnterZone, Inc

If you were a dial-up user, chances are you wouldn't be able to do that.

REALLY? Lets check this out.

A few simple reasons come to mind: first, you wouldn't have any or not enough disk space on your system account (limited by quota) to store the file.

Have you thought about that before sending a large file via email to someone? Many provides include your email spool in your quota. Beyond that, there are TONS of free hosting providers out there so your arguement there is moot.

Second, an average user probably wouldn't have the skill.

Huh? You're joking, right? Believe it or not Mitch, the rest of the internet population isn't just sitting around sucking up oxygen from brain-children like you. If they're competent enough to create a large presentation, they're competent enough to upload it somewhere. They can drag and drop with any number of FTP applications. Moot arguement.

Third, a .zip file will usually display as funny characters on a web browser - that's why ftp is needed.

Only if they're running REALLY, REALLY old browsers or the server itself is sending an incorrect mime-type for the .zip extension. Beyond that, they can right-click on the link and do a "save-as" so, this one is moot as well.

Fourth, you probably wouldn't have shell access and ftp space from your provider with a regular account.

Please see "free hosting providers" above. MOOT.

Fifth, assuming you would have all the toys, you would have to spend yourself the time to first upload the file, so that another may retrieve it.

OK. How is that any different that the time it takes you to send the file to the SMTP server? MOOT!

Sixth, if your file was a sensitive document, others would have public access to it, etc.

Ever hear of .htaccess? It's REALLY neat. If you think your file is safe from prying eyes in email, you've got more problems than not understanding basic authentication on a webserver though. You should stop argueing your invalid, moot points and spend that precious time reevaluating your security policy.

So what's a regular user to do? Email it!

No. That's what the uneducated newbie does. The regular user uploads it to their http/ftp server and sends a link to the file via email.

Hence the legitimate use of email for transmission of large files.

Please don't breed.

Most ISPs know that if they start limiting this privilege, users will migrate to someone that allows it.

If you educate your users, you have no problems. --- John Fraizer EnterZone, Inc

While that may be the case for a large number of people, nobody involved in this discussion is that clueless. Hint: they'd probably have an aol.com or msn.com email address, and no clue what NANOG is.

As I stated in a previous posting, ignorance doesn't always get people off the hook as a reason not to do things correctly. It's very easy to explain to someone how to use HTTP vs sending messages by email.

John, here's the steps involved in sending the file to the SMTP server:

Click "attach".

Select the appropriate directory.

Click on the file.

Or are you assuming that the folks in this discussion don't have LANs, and are using uuencode and piping through /bin/mail? I think your information is about 15 years out of date.

It's just as easy to make a shortcut to something like: ftp://user:pass@users.mywebspace.com/ Drag and drop file into browser. Simply type hyperlink into email. ie: http://users.mywebspace.com/user/

And yet, I continue to exchange files with other system administrators of Fortune 500 companies. Guess we're all "uneducated newbies".

Files via email is not completely evil, it has it's uses, however, more people abuse it than just simply use it on occasion to send a small attachment. The point I believe of this argument are the morons that attach multi-megabyte files which waste system resources on servers, waste bandwidth and generally stalls other emails from being delivered. -- Robert Blayzor IP Network Engineer, BOFH BiznessOnline.com, Inc. rblayzor@thebiz.net noc@thebiz.net http://www.thebiz.net/ FreeBSD, Putting the 'Operating' back into OS! -- http://www.freebsd.org/

On Fri, May 25, 2001 at 12:49:56PM -0400, Mitch Halmu exclaimed: [John Frazier]

...

Have you thought about that before sending a large file via email to someone? Many provides include your email spool in your quota. Beyond that, there are TONS of free hosting providers out there so your arguement there is moot.

And many don't. Remember, it has to work for all.

NOTHING 'works for all'. Nothing. End of story. If you're going for 100% functionality, you might as well give up now.

...

Huh? You're joking, right? Believe it or not Mitch, the rest of the internet population isn't just sitting around sucking up oxygen from brain-children like you. If they're competent enough to create a large presentation, they're competent enough to upload it somewhere. They can drag and drop with any number of FTP applications. Moot arguement.

Ever tried to explain to Joe user what FTP is? DeeAnn Mikula answered that one to the point.

As many others have pointed out ... Joe User, if s/he knows how to use a mail client, almost certainly knows how to user a browser. Problem solved.

...

Only if they're running REALLY, REALLY old browsers or the server itself is sending an incorrect mime-type for the .zip extension. Beyond that, they can right-click on the link and do a "save-as" so, this one is moot as well.

Some still do. Just pointing out potential problems.

The fraction of people both 1) clueless enough to not understand how to click a link, or upload a file; and 2) running software so old as to not facilitate this is almost nil. Besides the fact that those belonging to set (2) are much more likely to be clueful. I use a text-based email client and browse with lynx frequently, but it's not because I don't know how to install newer software or upload a file.

...

Please see "free hosting providers" above. MOOT.

Well, for one, free stuff on the Internet is comatose. http://www.dotcomeon.com/free.html

http://www.geocities.com http://www.tripod.com http://www.xoom.com http://clubs.yahoo.com and the list goes on and on and on and on ... just look at the free porn sites sometimes. They pop up like mushrooms, almost always hosted on free accounts that last for a month or so and then move elsewhere. There are more free webhosts than one can list.

Second, don't tell me that a user should be required to have both a paid account and a freebie just to be able to handle large files.

A user doesn't. If a user has an account they are paying for, and unable to upload a file through either ignorance or restriction by their ISP, they need to get another account. Any ISP's front-line support should be able and willing to give basic instructions to paying customers on how to utilize the technology they're paying for.

...

OK. How is that any different that the time it takes you to send the file to the SMTP server? MOOT!

Some may compose a message off-line, and have it sent unattended with other unsent messages when they go online.

How is this different than uploading the file when you go online? Or composing the message while offline, and when you send the message, uploading the file?

...

Ever hear of .htaccess? It's REALLY neat. If you think your file is safe from prying eyes in email, you've got more problems than not understanding basic authentication on a webserver though. You should stop argueing your invalid, moot points and spend that precious time reevaluating your security policy.

Joe user doesn't know about .htaccess, nor does he care. He just wants his document sent out like now! Joe doesn't see why he should jump through more hoops than his buddy on a competing service anyway.

I know this I'm fighting a losing battle, but there's a reason that FTP stands for File Transfer Protocol. I'll stop replying now, and return to my corner to be annoyed at the drop in the average clue level around the Net. Just because we _can_ do a thing, it does not necessarily follow that doing so is a good idea.

Besides, you will now have to confirm that the intended party retrieved the document, then go back in and delete it.

What confirmation? How do you know the intended party retrieved your 10MB email successfully? You get a reply, right? And you get a reply when they have downloaded the file, too. And users need to learn not to leave huge temp files lying around anyway. Deleting a file via FTP is hardly a technical challenge, even for the uninitiated.

...

No. That's what the uneducated newbie does. The regular user uploads it to their http/ftp server and sends a link to the file via email.

We do provide 10MB of personal web space with every account since 1995. Guess how many users even have web pages up? Many simply don't care.

Not having the motivation, knowledge or creativity required to build a website, and being unable to upload a file are two entirely different situations.

...

Please don't breed.

Some other brave souls dared to disagree.

Everybody has the right to their own opinion. Even if it's wrong. :)

John, we provide a service, and don't run a training camp. Most people wouldn't agree to the punishment you want to subject them to anyway.

If education is viewed as punishment, it's generally due to the experience provided by the educator.

--Mitch NetSide

-- Scott Francis scott@ [work:] v i r t u a l i s . c o m Systems Analyst darkuncle@ [home:] d a r k u n c l e . n e t West Coast Network Ops GPG keyid 0xCB33CCA7 illum oportet crescere me autem minui

Sabri Berisha wrote:

Then he/she should learn. I don't buy a car if I can't drive. I'm sorry for comparing internetworking with driving a car but I feel that FTP'ing is a basic skill if you want to use the internet in a professional way (and since most documents are being distributed for professional reasons, they should know).

Another option is to pay someone to drive you around. If you look into it, you can see that an established market for electronic document exchange services already exists. The most established company doing this is probably UPS. See the UPS Document Exchange at <http://exchange.ups.com/> (Disclaimer: I have no affiliation whatsoever with UPS) --

...

Magenta Sites Marko Karppinen

On Fri, May 25, 2001 at 04:32:01PM +0100, David Howe exclaimed:

Hmm. The problem with this analogy is the extension "I know you can drive a car; however, you really should learn to drive a truck as it is much more efficient for large loads" which is of course true.......

and when the user tries to transport a ton of <foo> in their yugo, we see the following result: http://www.tardsite.com/trdomnth122000.htm :) -- Scott Francis scott@ [work:] v i r t u a l i s . c o m Systems Analyst darkuncle@ [home:] d a r k u n c l e . n e t West Coast Network Ops GPG keyid 0xCB33CCA7 illum oportet crescere me autem minui

Amen... Look at companys like logitech who create programs to email your grandma you kids baby pictures. Your gonna tell grandma she needs to FTP that file from her daughter FTP sites. And tell daughter she has to set it up. Daughter has no concept of bandwidth and doesn't want one. I think we often forget we work for the users and if they want to email attachment and it's not hurting anyone else so be it. I am happy she can do it... Have you ever seen USENET if you wanna talk about a waste of bandwidth by UUencoding files you should start there first, email is the least of your problems in this arena.... one ten meg attachment on an email isn't anything compared to the alt.binaries traffic. Yet no one has an issue with that..... When it comes down to it, what ever is easier is what the average joe is gonna do. Dropping a file in the attachment window of my email client is 10 times easier than setting up an FTP/Web server to upload your file too. Lets follow the basic steps. Email Way: 1) open email client 2) type address 3) drag and drop file 4) click send FTP way: 1) open email client 2) type address 3) type in link 4) open FTP program 5) open connection 6) upload file 7) click send it's almost twice as many steps, from a human factor prospective the email way is much more favorable. Remember computer work for us we don't work for computers. Considering the one downside or inflated file size it's much more preferable for myself. I dunno about you but my boss doesn't want to ftp my reports, he want to click on an icon in his email. Rob deeann mikula wrote:

On Fri, 25 May 2001, Sabri Berisha wrote:

...

...

Second, an average user probably wouldn't have the skill.

Then he/she should learn. I don't buy a car if I can't drive. I'm sorry for comparing internetworking with driving a car but I feel that FTP'ing is a basic skill if you want to use the internet in a professional way

uh, when is the last time that you worked the front line of an ISP? most people have never even heard of FTP. i almost spewed my orange juice all over my desk when i read that paragraph. :)

unfortunately, i have accepted the fact that there is no stopping customers from emailing 10MB (and larger!) files, they can just go to our competition if we prevent it.

i bitch, moan, and insist that the front line kids give the "email != FTP speech," but we continue to permit it. *sigh*

unless we go back in time and re-do the protocols, and re-educate newbies before setting them lose on the 'net, i don't think that we are going to stop people from emailing what should be FTP'd.

deeann m.m. mikula director of operations

telerama public access internet http://www.telerama.com 1.877.688.3200

At 06:18 PM 5/25/01 -0400, Steve Sobol wrote:

Apples != oranges. You can avoid reading the binaries groups, and server admins can refuse to carry them.

Once an e-mail is dumped in your mailbox, you can't automagically delete it if you are Joe Sixpack Average NetUser. You either have to go in with your e-mail client and download and delete it, or ask your ISP to nuke the message.

*My* experience is that people who weren't expecting the big files from the yahoos that send it to them think their e-mail is broken until they call and I check their mail spool and see there's a big file (obviously the situation is slightly different if they were expecting the message, but there are a lot of times when I hear that someone's friend or relative just dumped a big mail in their box). Large files are an annoyance to the customer in cases like that.

That's why the big-email dilemma is such a... well, such a dilemma. If you disallow huge emails they complain. If you allow them, they get stuck and need help. It was even worse when every 1user was using Outlook 97, which routinely choked on any email over 500K. We (my former employer) finally started charging $10 to clear out a stuck mailbox. After a time or two they would learn to use the web interface and clear it out themselves. A technical solution (like the one mentioned earlier) would be great, but it would need to work both ways, otherwise it wouldn't help until many people implemented it. Converting outgoing attachments to a link would be great, but it would also need to convert incoming attachments to a link. Incoming is the worse problem IMO. I'll add that to the list of things I plan to work on if I find myself without 60 hours/week of work that I get paid for.

At 02:01 PM 5/25/01 -0400, Greg A. Woods wrote:

Let's inject some semi-operational content into this thread.

Number 1: You *want* the idiots who mail large files to go to the competition!

I wish that were true. I don't want 80% of my residential clients to go away. It's extremely difficult to underestimate the intelligence of the average residential 1user. How much can you expect from people who start their business relationship with you by asking "Can you put the Internet on my computer?" In a business environment, the idiots don't go away. You do.

Number 2: Do something about it! I regularly configure mail servers to a maximum of 4MB, and usually to just 100KB. Anyone who "complains" is told to upload their file to the web server and e-mail the URL. The complainant usually then says "Heh! Why didn't I think of that?!?!?", and if your front-line tech support people are really good they won't laugh back at the customer (at least not while he or she is still on the line ;-).

That must be nice. Do you provide service to Mensa members? My customer service people were inundated with angry calls when I tried limiting it to 1MB. It's not cost-effective to pay someone $10+/hour to teach hundreds of people to use FTP, especially when they aren't interested in learning.

If you make it impossible for them to e-mail large files (except by breaking them down into a gazzillion little e-mails) then you'll be well past the gate and all smart people will be stampeeding to use your HTTP and FTP servers for what they're meant for! (The not-so smart ones will need to be told once to do that, and if you're lucky the idiots will immediately switch to your competition and drive them into the ground by clogging their mail servers *and* their tech support lines.)

It's clearly obvious that emailing large files is evil, but if you're in the business of providing Internet access to end-users, you must accept the fact that the vast majority of your clients are computer-illiterate. They don't know how to FTP, and they don't want to know. If they can't email the dancing bears to grandma (and 20 other friends) they will switch to the competition and clog *their* bank account with money. I tried limiting filesizes to 1MB, then 2MB, etc... At 10MB, the complaints got down to a manageable level. I found that it makes more sense to throw hardware at the problem. Hardware is a 1-time cost. Lost clients cost you money every month, and they will tell their friends that you suck. When Adlai Stevenson was running for president, one of his advisors said "We have all of the thinking people on our side." He said "That means that we are going to lose!" It would be nice to only sell service to non-morons, but I don't think it would be profitable. If you get a guy who constantly emails 10M files to 40 people, it makes sense to get rid of him, but you really do have to cater to 1users if you sell residential Internet service. In a business environment, you will find that computer literacy is inversely proportional to seniority. If you do things that make sense technically, without considering how they will affect idiots who want to stupid things, you won't last long.

On Fri, May 25, 2001 at 11:22:15AM -0400, deeann mikula wrote:

On Fri, 25 May 2001, Sabri Berisha wrote:

...

...

Second, an average user probably wouldn't have the skill.

Then he/she should learn. I don't buy a car if I can't drive. I'm sorry for comparing internetworking with driving a car but I feel that FTP'ing is a basic skill if you want to use the internet in a professional way

uh, when is the last time that you worked the front line of an ISP? most people have never even heard of FTP. i almost spewed my orange juice all over my desk when i read that paragraph. :)

yep, and thanks to people like you, they will never hear about FTP. you are helping to create a society of users who think that Netscape/Explorer is the internet, with no concept of any of its other applications.

unfortunately, i have accepted the fact that there is no stopping customers from emailing 10MB (and larger!) files, they can just go to our competition if we prevent it.

that would be my tactic, but then i'm my own boss, so i can make those decisions. i get the impression that you (or someone else on the "pro-ftp-by-email" side) is "just and employee" and has to do as they are told. well, have a bit of guts, and try to explain to management what the implications (both current and local as well as future and far-reaching) of such policies are. when 100 customers leave because 1 customer hosed your mail server, then maybe they will start listening to tachnical advice, as opposed to relying solely on marketing advice. -- [ Jim Mercer jim@reptiles.org +1 416 410-5633 ] [ Now with more and longer words for your reading enjoyment. ]

On Fri, 25 May 2001 16:03:21 PDT, Scott Francis said:

...

that's why ftp is needed. Fourth, you probably wouldn't have shell access and ftp space from your provider with a regular account. Fifth, assuming

you don't NEED shell access or ftp space! Just click the link in your email, or copy/paste to your browser, and *poof* the file is on your local machine.

I do believe he meant that "you as the user *sending* the file need shell access and FTP space". And there's something that people are overlooking here - the *clean up* afterwards. Sure - I can upload the file to a server and send an e-mail with a pointer to it. BUT WHEN AND HOW DOES THAT FILE GET CLEANED UP? This is particularly interesting in the case of multiple recipients. Just yesterday, I had to send a draft of a white paper out to some 25 people who wanted to review it. Point, click, attach, send - and now I am *fairly* confident that unless I get a bounce message back, that all 25 people have a copy (no, it's not perfect, but things usually work quite well - it either arrives or bounces - mail evaporating is quite rare in the global scheme of things). Now let's say I had uploaded it to a server. When do I know that it's safe to remove it? If I wait 3 days and remove it, I've possibly screwed somebody over. Forget that - I'll just leave it on the server indefinitely. So now instead of needing 1M of spool space for the few minutes it takes to send it out, I'm tying up 1M of spool space until I clean it up by hand. Somehow, this is just *screaming* a "tragedy of the commons" scaling failure at me ;) Oh - and did I mention that the prior business relationship with most of these 25 was a business card from last week's SANS? This means that if I had uploaded it to a server, I'd have to worry about how to grant access rights for that document. I'm lucky that there's nothing confidential in *that* white paper - I'm pretty sure that I'd hit any number of snags and screw-up trying to put a .htaccess restriction that actually worked.... Valdis Kletnieks Operating Systems Analyst Virginia Tech

On Sat, 26 May 2001 13:15:29 EDT, Greg A. Woods said:

The space occupied on the HTTP (or FTP) server isn't "spool space". It's already destined as "archive space" (many ISPs don't even back it up and state flatly that it's the user's responsibiilty to keep archive copies in case of disaster or whatever).

OK. So the thing lives in /tempurls instead of /var/spool. The point was that quantity X of disk was going to be needed for at least a week or to, instead of probably just a few minutes. You get 100K customers all using several X of disk for a week instead of a few minutes, you better be planning on more disk..

Of course an ISP might provide a few GB of cheap disk for "temp" URLs in people's home pages that'd get cleaned up automaticaly after a few days/weeks/whatever of inactivity. (http://my.isp/~me/temp/ is a symlink/alias/whatever created by the ISP into this temp space)

My point was that you may need more than just a few GB of cheap disk for this.

Don't you know how to encrypt files? Hopefully any confidential e-mail message you send is encrypted so why wouldn't you encrypt the external-body content with the very same key before uploading it to a public server?

Hmm.. OK.. so I can either use a symmetric key scheme and figure out how to distribute 25 copies of the key for this file, or I use public key scheme and encrypt each one to the recipient's public key (and park 25 seperate copies on the server, aggrivating the problem I mentioned above). Did I mention that: a) PKI is still a pipe dream b) I'm *well* aware of how to encrypt files - I'm the guy who is continually being asked what the <bleep> an application/pgp-signature is. c) this infrastructure is getting more and more complicated.. ;) As I said - I'm glad the draft wasn't proprietary information.

...

I'm pretty sure that I'd hit any number of snags and screw-up trying to put a .htaccess restriction that actually worked....

You certainly would not want to do anything so complex and pointless.

Especially not when just PGP'ing the mail and sending it already does all the necessary securing of the infrastructure.... -- Valdis Kletnieks Operating Systems Analyst Virginia Tech

Craig Partridge wrote:

Part of this discussion is just plain bizarre.

It is worth remembering that SMTP is, in most respects, simply FTP reworked. In many ways, HTTP is FTP badly reinvented.

I disagree - HTTP is more firewall/NAT friendly, and has no active/passive mode.

But for a little extra SMTP handshaking at the start, there is no efficiency difference in transfer rate between SMTP and FTP. Probably the same is

No, there is overhead in encoding of binary data for transmission by SMTP.

true for HTTP though I've not looked.

Jan

In message <GNEIJMGJNDJMNEGPDBGBAEJGDHAA.rblayzor@thebiz.net>, "Robert Blayzor" writes:

I think you missed the fact that sending files via SMTP is incredibly inefficient. Any files sent via SMTP have to be encoded which can balloon the transmission up 30%+. That is an incredible waste of bandwidth on a 10MB file. Also, remember that SMTP usually relays, so the message is bounced between 1-8 servers along the way (or more), more bandwidth and resources wasted. *sigh*

I've gotten a bunch of notes on this topic. Issues in order: * Email encoding is inefficient. It doesn't have to be. A zipped uuencoded file is often smaller than the source file and rarely longer. Why not update the MIME standards to encourage compression of binaries? This is the network operators mailing list -- you can certainly go to IETF with operational concerns and have credibility. Then we could block attachments that don't implement the new encoding and, hey, actually improve the world! * SMTP usually relays. Yes it often does. Typically you'll relay a couple of times. But most of those relays are at high bandwidth locations with lots of disk space -- they're not suffering. * A POP site may find itself storing 200 copies of the same binary. That's true, and a problem. There's an obvious solution: do what mail daemons do and share the file among mailboxes, but that solution increases risk of corruption (e.g. the pointer to the file gets trashed and you retrieve the wrong attachment). In short, I'm not sympathetic with the first two concerns. Craig

...

But for a little extra SMTP handshaking at the start, there is no efficiency difference in transfer rate between SMTP and FTP. Probably the same is true for HTTP though I've not looked.

I think you missed the fact that sending files via SMTP is incredibly inefficient. Any files sent via SMTP have to be encoded which can balloon the transmission up 30%+. That is an incredible waste of bandwidth on a After all, nmodep compress it back to the original size while transferring. The rate loss is 5 - 10%, not 30 - 40.

For LAN, this volumes are not large (1 - 2 Mb).

10MB file. Also, remember that SMTP usually relays, so the message is bounced between 1-8 servers along the way (or more), more bandwidth and resources wasted. *sigh*

-- Robert Blayzor IP Network Engineer, BOFH BiznessOnline.com, Inc. rblayzor@thebiz.net noc@thebiz.net http://www.thebiz.net/

FreeBSD, Putting the 'Operating' back into OS! -- http://www.freebsd.org/

On Fri, 25 May 2001 18:12:30 EDT, Steve Sobol said:

Perhaps there is no inherent difference in throughput, but the files are necessarily larger when sent through e-mail, as they must be base64 encoded.

There's no *inherent* reason they *must* be base64 encoded. Please note that all the way back in RFC1341, in addition to a CTE of 'base64', there's the '8bit' and 'binary' CTEs. Sendmail started having sane support of '8bit' way back in 8.7, in 1995. RFC3030 defines the BDAT extension for SMTP. Now, the major real-life reason why encoding is needed is because Sendmail 8.12 still doesn't have BDAT support. If there's sufficient real-world demand, it's probably implementable fairly easily (a quick readig of RFC3030 and the Sendmail 8.12 source doesn't pop out any astounding show-stoppers)... If there's a lot of demand for it, I'll see what it would take to get it onto the Sendmail 8.13 feature request list.... Valdis Kletnieks Operating Systems Analyst Virginia Tech

On Sat, 26 May 2001 10:48:48 EDT, Mitch Halmu said:

Besides interoperability concerns with current software that will have to become 'legacy', you will open the gates to any hacker and script kiddie that can now mail you their favorite virus, trojan or worm just as is was compiled. It's bad enough that unprintable characters and buffer overflows in the header must be neutered. Scary stuff that you haven't even thought of could happen with 8 bit message bodies...

FUD. Complete and total FUD. There's *NO* change in what can be transferred. The ONLY difference is you can avoid converting it to base64 at the one end and then decoding it at the other end. If you haven't noticed, viruses, trojans, and worms are being transmitted just as were compiled, via the *current* infrastructure. And the need for neutering characters in the header was recognized in RFC1342, 10 years ago. And 8 bit message bodies have been supported by Sendmail since 8.7, all the way back in 1996. The only thing that BDAT buys you over the currently existing and *widely* deployed 8bitmime support (where Sendmail will even automatically upgrade/downgrade from 8bit to 7bit using either quoted-printable or base64, on a configurable basis), is that 8bitmime is still bound by the 1000-char line length restriction in SMTP, where you need a CR-LF pair at least every 998 characters. BDAT doesn't open any exposures other than programmers that can't get a length-data encoding right. Hmm.. maybe we *should* worry... ;) -- Valdis Kletnieks Operating Systems Analyst Virginia Tech

On Sat, 26 May 2001 15:46:56 EDT, Mitch Halmu said:

Hmmm, I'm looking at an encoded snowhite message body right now. midgets.scr encoded in base64, and transmitted as an attachment. Can provide you a copy in private if you want to take it apart (but not on a PC, or you'll get a *huge* surprise ;)

Notice the surprise isn't when your broken MUA decodes it from base64 to binary. The surprise is when your broken MUA then takes that binary and does something stupid with it.

All others in that family that I looked at were also encoded. Did anyone get a raw binary via regular email?

And if you pay any attention - it's *NOT* the base64 decoding that protects you from these things - it's HAVING AN MUA THAT ISN'T STUPID ABOUT RUNNING EXTERNAL CODE. -- Valdis Kletnieks Operating Systems Analyst Virginia Tech