They might as aleron used to offer it. That comes with the disclaimer that I have never tried it. John :) -----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of Drew Weaver Sent: Tuesday, September 05, 2006 9:10 AM To: nanog@merit.edu Subject: NNTP feed. Does anyone know if cogent offers NNTP feeds to their DIA customers? Before we take the plunge we need to know and the sales fellas werent able to tell me this. -Drew
What is the current BCP to establish a well-connected news server nowadays? All the guys I used to know who were experts in this... um, don't run news servers anymore. :) If you want to privately offer me an NNTP feed that would be welcome -- we'll even peer with you because of it. Thanks Deepak John van Oppen wrote:
They might as aleron used to offer it. That comes with the disclaimer that I have never tried it.
John :)
-----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of Drew Weaver Sent: Tuesday, September 05, 2006 9:10 AM To: nanog@merit.edu Subject: NNTP feed.
Does anyone know if cogent offers NNTP feeds to their DIA customers? Before we take the plunge we need to know and the sales fellas werent able to tell me this.
-Drew
we don't run one either... :) The last person I know who was running one, was in the proccess of killing it. john :) -----Original Message----- From: Deepak Jain [mailto:deepak@ai.net] Sent: Tue 9/5/2006 3:37 PM To: John van Oppen Cc: Drew Weaver; nanog@merit.edu Subject: Re: NNTP feed. What is the current BCP to establish a well-connected news server nowadays? All the guys I used to know who were experts in this... um, don't run news servers anymore. :) If you want to privately offer me an NNTP feed that would be welcome -- we'll even peer with you because of it. Thanks Deepak John van Oppen wrote:
They might as aleron used to offer it. That comes with the disclaimer that I have never tried it.
John :)
-----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of Drew Weaver Sent: Tuesday, September 05, 2006 9:10 AM To: nanog@merit.edu Subject: NNTP feed.
Does anyone know if cogent offers NNTP feeds to their DIA customers? Before we take the plunge we need to know and the sales fellas werent able to tell me this.
-Drew
John van Oppen wrote:
we don't run one either... :)
The last person I know who was running one, was in the proccess of killing it.
Apparently you found some people killing it off, while there are actually companies who specialize in NNTP access. It seems that for mysterious reasons which the RIAA and other such organizations apparently don't seem to understand that these companies are also causing quite a lot of traffic to be shifted over the internet. Peeking at for instance http://www.nextfeed.nl/ reveals that there is one ISP having 40 days retention which apparently maps to 6*50 TB (that is 300 Terabytes indeed) of storage space, while there are also another having 50 days of retention, most likely mapping to somewhat like 400 Tb. On average they seem to be shifting in the vicinity of 15Tb/day though, looking at the number 14 of the top1000.org list. For hardware freaks it of course gives some nice things like the dutch newszilla installation: http://wa.ter.net/gallery2/images/newszilla That single setup already makes quite some small hosting companies drool out of both corners ;) Networking freaks will love the "Core Juniper 640 handles newszilla traffic" comment <grin> Otherwise said: if you are setting up a full-nntp-feed capable box, you'll have to dig nice and deep into that money bag but on the other hand there seems to be loads of people doing a lot of posting and reading, where else would the volume of that traffic come from? For the people trying to find peers, check: http://www.usenet.com/peering/peeringpage.cfm and of course also: http://www.top1000.org/ where even google pops up in a 4th place. Greets, Jeroen
On Wed, 6 Sep 2006, Jeroen Massar wrote:
seems to be loads of people doing a lot of posting and reading, where else would the volume of that traffic come from?
I guess experiences differ from different organisations, when I discovered that server-server traffic was at least 10x more than people actually read (server-client) I didn't feel like trying to get my (then) employer continue running the NNTP server. My feeling today (or rather, 3-5 years ago) is that NNTP is used instead of bittorrent and other PtP protocols to move copyrighted material, and I'd say that it probably makes more sense for some to let their users invest in that drivespace than to have themselves run an NNTP server and spend operational resources on keeping it running well. I've also heard people complaining about a few NNTP users causing a lot of helpdesk tickets in respect to "single message missing" because they cannot download that 4.7 gig ISO correctly because a message got lost somewhere in the middle. -- Mikael Abrahamsson email: swmike@swm.pp.se
On Wed, 6 Sep 2006, Mikael Abrahamsson wrote:
On Wed, 6 Sep 2006, Jeroen Massar wrote:
seems to be loads of people doing a lot of posting and reading, where else would the volume of that traffic come from?
I guess experiences differ from different organisations, when I discovered that server-server traffic was at least 10x more than people actually read (server-client) I didn't feel like trying to get my (then) employer continue running the NNTP server.
My feeling today (or rather, 3-5 years ago) is that NNTP is used instead of bittorrent and other PtP protocols to move copyrighted material, and I'd say that it probably makes more sense for some to let their users invest in that drivespace than to have themselves run an NNTP server and spend operational resources on keeping it running well.
I've also heard people complaining about a few NNTP users causing a lot of helpdesk tickets in respect to "single message missing" because they cannot download that 4.7 gig ISO correctly because a message got lost somewhere in the middle.
I came to much the same conclusion several years ago, when we finally decommissioned our NNTP Servers and out-sourced the service to an outside company. Running an NNTP server was a full-time job, and the 500 or so people that used it didn't generate enough revenue for us to continue managing it inside. -- Vice President of N2Net, a New Age Consulting Service, Inc. Company http://www.n2net.net Where everything clicks into place! KP-216-121-ST
Thus spake "Greg Boehnlein" <damin@nacs.net>
I came to much the same conclusion several years ago, when we finally decommissioned our NNTP Servers and out-sourced the service to an outside company. Running an NNTP server was a full-time job, and the 500 or so people that used it didn't generate enough revenue for us to continue managing it inside.
That seems to be consensus among ISPs: there just aren't enough users today to justify the cost of hiring a full-time news admin, deal with abuse and customer service issues, and pay for the storage space for 8TB/day of content. OTOH, it might be doable if you didn't carry the alt.binaries groups; those account for well over 90% of the bytes on usenet today, and virtually all of the complainers. If your customers want the binaries groups, you can easily point them to a dozen different commercial news providers that do carry them -- and have the economy of scale needed to turn a profit. Another option is to run a caching-only news server, provided you can find a willing upstream. This will save you most of the bandwidth if you don't have many users, though again excluding binaries may be needed to cut down on the whiney pirates. (Besides, all the binaries on usenet are available via BitTorrent somewhere anyways; NNTP does not make a good piracy protocol from a technical perspective, only from an anonymity one) S Stephen Sprunk "God does not play dice." --Albert Einstein CCIE #3723 "God is an inveterate gambler, and He throws the K5SSS dice at every possible opportunity." --Stephen Hawking
On Tue, 5 Sep 2006, Stephen Sprunk wrote:
(Besides, all the binaries on usenet are available via BitTorrent somewhere anyways; NNTP does not make a good piracy protocol from a technical perspective, only from an anonymity one)
Do you believe anonymity has a low intrinsic value for internet applications? matto --matt@snark.net------------------------------------------<darwin>< Moral indignation is a technique to endow the idiot with dignity. - Marshall McLuhan
"Stephen" == Stephen Sprunk <stephen@sprunk.org> writes:
[NNTP servers] Stephen> OTOH, it might be doable if you didn't carry the Stephen> alt.binaries groups; those account for well over 90% of the Stephen> bytes on usenet today, More like 99.9%, and an even larger proportion if you filter out binaries from other groups too. Broadly speaking the daily volume is something like 2.8TB/day for the whole feed, or somewhere in the ballpark of 1GB/day for the non-binaries feed. Stephen> Another option is to run a caching-only news server, Stephen> provided you can find a willing upstream. If you're talking about binaries, the fact is that Usenet simply does not cache well. You'll have to use many terabytes of disk to get even a modest hit-rate, and the performance benefits will be slight. For text, there's no point in running a caching server rather than a real one, since the incoming bandwidth for text is so small (couple of hundred kbits) and the storage requirements so modest (one 36GB spool disk will hold a month's traffic). -- Andrew, Supernews http://www.supernews.com -- individual and corporate NNTP services
I guess I should say that most people are outsourcing to the bigger news shops (at least the people I know are) due to the hardware demands of today's news volumes. john :) -----Original Message----- From: Jeroen Massar [mailto:jeroen@unfix.org] Sent: Tue 9/5/2006 4:10 PM To: John van Oppen Cc: deepak@ai.net; Drew Weaver; nanog@merit.edu Subject: Re: NNTP feed. John van Oppen wrote:
we don't run one either... :)
The last person I know who was running one, was in the proccess of killing it.
Apparently you found some people killing it off, while there are actually companies who specialize in NNTP access. It seems that for mysterious reasons which the RIAA and other such organizations apparently don't seem to understand that these companies are also causing quite a lot of traffic to be shifted over the internet. Peeking at for instance http://www.nextfeed.nl/ reveals that there is one ISP having 40 days retention which apparently maps to 6*50 TB (that is 300 Terabytes indeed) of storage space, while there are also another having 50 days of retention, most likely mapping to somewhat like 400 Tb. On average they seem to be shifting in the vicinity of 15Tb/day though, looking at the number 14 of the top1000.org list. For hardware freaks it of course gives some nice things like the dutch newszilla installation: http://wa.ter.net/gallery2/images/newszilla That single setup already makes quite some small hosting companies drool out of both corners ;) Networking freaks will love the "Core Juniper 640 handles newszilla traffic" comment <grin> Otherwise said: if you are setting up a full-nntp-feed capable box, you'll have to dig nice and deep into that money bag but on the other hand there seems to be loads of people doing a lot of posting and reading, where else would the volume of that traffic come from? For the people trying to find peers, check: http://www.usenet.com/peering/peeringpage.cfm and of course also: http://www.top1000.org/ where even google pops up in a 4th place. Greets, Jeroen
On Tue, 5 Sep 2006, John van Oppen wrote:
we don't run one either... :)
The last person I know who was running one, was in the proccess of killing it.
I used to run one, but haven't, since about 2000 :) The provider i worked for at the time got out of the game and outsourced news because otherwise we would have needed to dump a lot of money into disk space to keep people from bitching about the sub-24 hr retention times on alt.binaries.$VICE :( That was the blessing and the curse of cyclical filesystems, I guess :) On the plus side, Highwind's news server software just flat-out ran - never gave me a single problem in the 3 years I ran it. That plus a separate box running diablo to manage the feeds was a winning combination :) More and more people are outsourcing news because providing good news service requires tons of disk space and loads of network bandwidth, i.e. it costs a lot of money to operate, and for many providers, it doesn't make any money. The last time I looked at newsfeed stats, a full feed with all the alt.binaries crap was running around 350 GB/day - I wouldn't be at all surprised if it's substantially higher now. jms
On Tue, Sep 05, 2006 at 10:27:29PM -0400, Justin M. Streiner wrote:
providing good news service requires tons of disk space and loads of network bandwidth,
I'm getting the impression that providing good news service doesn't need that, only providing good warez service does (and this includes all copyrighted pictures/video/audio media binaries). If folks would end abusing NNTP for file distribution via flooding, the matter would quickly be resolved. Am i naive?
The last time I looked at newsfeed stats, a full feed with all the alt.binaries crap was running around 350 GB/day - I wouldn't be at all surprised if it's substantially higher now.
And prolly 349GB/day being exactly the crap you mention. Best regards, Daniel -- CLUE-RIPE -- Jabber: dr@cluenet.de -- dr@IRCnet -- PGP: 0xA85C8AA0
On Wed, 6 Sep 2006, Daniel Roesen wrote:
If folks would end abusing NNTP for file distribution via flooding, the matter would quickly be resolved. Am i naive?
The technical term might be "trolling" . Binaries have made up the vast majority of Usenet bandwidth since at least the early 90s so hoping it will go away will not happen. One of the attractions to Usenet is that since the majority of messages (by number) are in fact discussions it is hard to claim it is an illegal file sharing network which is why (IANAL) the *AA hasn't shut it down yet and Usenet providers operate openly. Talking of "resolving" and "abusing" wrt binary newsfeeds grew old around 10 years ago. These days plenty of people run text-only feeds and leave the full-binary feeds to the top hundred or so sites that can afford them. -- Simon J. Lyall | Very Busy | Web: http://www.darkmere.gen.nz/ "To stay awake all night adds a day to your life" - Stilgar | eMT.
On Wed, Sep 06, 2006 at 04:29:54AM +0200, Daniel Roesen wrote:
If folks would end abusing NNTP for file distribution via flooding, the matter would quickly be resolved. Am i naive?
There is a reason Usenet hasn't gone the way of Gopher, and I assure you it isn't because of the the copious spam, net kooks, and trolls. There is a certain type of content that people want, and I'll give you one guess what that is... If you don't carry that content, you'll probably be doing more traffic in backscatter on the IP space then you will in NNTP. :) -- Richard A Steenbergen <ras@e-gerbil.net> http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)
From: Richard A Steenbergen <ras@e-gerbil.net> Date: Wed, 6 Sep 2006 01:24:07 -0400 To: "nanog@merit.edu" <nanog@merit.edu> Subject: Re: NNTP feed.
On Wed, Sep 06, 2006 at 04:29:54AM +0200, Daniel Roesen wrote:
If folks would end abusing NNTP for file distribution via flooding, the matter would quickly be resolved. Am i naive?
There is a reason Usenet hasn't gone the way of Gopher, and I assure you it isn't because of the the copious spam, net kooks, and trolls. There is a certain type of content that people want, and I'll give you one guess what that is... If you don't carry that content, you'll probably be doing more traffic in backscatter on the IP space then you will in NNTP. :)
-- Richard A Steenbergen <ras@e-gerbil.net> http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)
Richard, What type of content is that :) Just pulling your chain. But seriously there are many of us that use NNTP for legal matters and the pleasure of communicating with one another with something other than email for many reasons. If you are looking to get into the NNTP business and use it as a revenue generator I think you would be better of outsourcing to some of the companies that already has been mentioned and some that I know of that provide an awesome service that has not been mentioned :) I subscribe to many groups that I try to read daily some health related, political, and of course technology focused. But if this is something you want as a value added service I have yet to see anyone be able to maintain the infrastructure. I have went through the process of outsourcing to a news provider previously and if you would like could make recommendations offline.
Sorry to reply to my own post, but after reading further into this thread, I saw that my estimation of "substantially higher than 350 GB/day" shows how long I've been out of the business of driving large news servers :) jms On Tue, 5 Sep 2006, Justin M. Streiner wrote:
On Tue, 5 Sep 2006, John van Oppen wrote:
we don't run one either... :)
The last person I know who was running one, was in the proccess of killing it.
I used to run one, but haven't, since about 2000 :) The provider i worked for at the time got out of the game and outsourced news because otherwise we would have needed to dump a lot of money into disk space to keep people from bitching about the sub-24 hr retention times on alt.binaries.$VICE :( That was the blessing and the curse of cyclical filesystems, I guess :)
On the plus side, Highwind's news server software just flat-out ran - never gave me a single problem in the 3 years I ran it. That plus a separate box running diablo to manage the feeds was a winning combination :)
More and more people are outsourcing news because providing good news service requires tons of disk space and loads of network bandwidth, i.e. it costs a lot of money to operate, and for many providers, it doesn't make any money.
The last time I looked at newsfeed stats, a full feed with all the alt.binaries crap was running around 350 GB/day - I wouldn't be at all surprised if it's substantially higher now.
jms
participants (13)
-
Andrew - Supernews -
Daniel Roesen -
Deepak Jain -
Elijah Savage -
Greg Boehnlein -
Jeroen Massar -
John van Oppen -
Justin M. Streiner -
Matt Ghali -
Mikael Abrahamsson -
Richard A Steenbergen -
Simon Lyall -
Stephen Sprunk