Hi, Many moons ago, we got a set of Akamai servers. Over the years I think they replaced every one of them at least once. Last August we got a another set of servers due to a move and now two of those three servers have failed. I still have the original server that started garlic.com in production after 11+ years so I know servers can last a long time. I don't understand why Akamai failure rates are so high Is anyone else seeing high failure rates of Akamai servers at their facilities? Roy
On Mon, 28 Nov 2005, Roy wrote:
Is anyone else seeing high failure rates of Akamai servers at their facilities?
We had 3 boxes for 5-6 years without a problem. Then one of them failed. We've since replaced that box 5-6 times in the last year. The replacement boxes often come with non-spining CPU fans and other issues so I'm not that surprised. The last replacement was a few months ago though so maybe this one will stick around. I think whoever is doing their refurbs isn't doing a very good job. They never seem very concerned though. Chris -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Chris Owen ~ Garden City (620) 275-1900 ~ Lottery (noun): President ~ Wichita (316) 858-3000 ~ A stupidity tax Hubris Communications Inc ~ www.hubris.net ~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
At 01:39 PM 11/28/2005, Roy wrote:
Hi,
Many moons ago, we got a set of Akamai servers. Over the years I think they replaced every one of them at least once. Last August we got a another set of servers due to a move and now two of those three servers have failed. I still have the original server that started garlic.com in production after 11+ years so I know servers can last a long time. I don't understand why Akamai failure rates are so high
Is anyone else seeing high failure rates of Akamai servers at their facilities?
Out of the total three Akamai servers we have, I think we've had two of them replaced in the past three or four years that we've had them. One was replaced several times. The replacement servers tend to be refurbished and I've seen multiple things wrong with them when they arrive. If I recall correctly, one replacement wouldn't even boot successfully... Just kept crashing. Reloading the OS from an Akamai recovery CD had no affect. Shipping does cause problems whereby the parts can come loose during transit. The most common problem we see is failed hard drives and/or SCSI bus errors which are likely related to the hard drive failures. I'm surprised Akamai doesn't have any hardware RAID with hot swap yet (at least not in the boxes we have). It would be much less costly for them to ship a new hard drive than a whole new server each time a hard drive fails. I know the idea is to have very cheap boxes in clusters, but I wonder how much they're paying in shipping for replacing the cheap hardware. As of late, we've had no known problems with our Akamai boxes. That one box does occasionally have weird SCSI hangs where the other two work nonstop. For the most part it is fine though. Vinny Abello Network Engineer Server Management vinny@tellurian.com (973)300-9211 x 125 (973)940-6125 (Direct) PGP Key Fingerprint: 3BC5 9A48 FC78 03D3 82E0 E935 5325 FBCB 0100 977A Tellurian Networks - The Ultimate Internet Connection http://www.tellurian.com (888)TELLURIAN "Courage is resistance to fear, mastery of fear - not absence of fear" -- Mark Twain
On Nov 28, 2005, at 2:02 PM, Vinny Abello wrote:
I know the idea is to have very cheap boxes in clusters, but I wonder how much they're paying in shipping for replacing the cheap hardware.
Never underestimate the amount of airbills that can be paid with KISS strategy. Anything else is trollage on NANOG.
On Mon, 28 Nov 2005, Christian Kuhtz wrote: > > I know the idea is to have very cheap boxes in clusters, but I wonder > > how much they're paying in shipping for replacing the cheap hardware. > Never underestimate the amount of airbills that can be paid with KISS > strategy. Yep, that's true. Shipping is cheap, it's customs that's expensive and time-consuming, and Akamai tends to avoid the kind of places where you have to deal with a lot of customs. -Bill
On Mon, 28 Nov 2005, Bill Woodcock wrote:
On Mon, 28 Nov 2005, Christian Kuhtz wrote: > > I know the idea is to have very cheap boxes in clusters, but I wonder > > how much they're paying in shipping for replacing the cheap hardware. > Never underestimate the amount of airbills that can be paid with KISS > strategy.
Yep, that's true. Shipping is cheap, it's customs that's expensive and time-consuming, and Akamai tends to avoid the kind of places where you have to deal with a lot of customs.
I'd note that the original poster didn't classify 'broken' or 'outage' or 'non-functioning'... just the end result: "replacement". So, is akamai doing some fancy SMART detection and seeing bad fans and replacing, seeing a bad cpu fan or disk or memory corruption and replacing, or are these hard box outages with no recourse but a complete immediate replacement? (just curious as they don't let us play with these pieces/parts :) )
On Mon, 28 Nov 2005, Christopher L. Morrow wrote:
On Mon, 28 Nov 2005, Bill Woodcock wrote:
On Mon, 28 Nov 2005, Christian Kuhtz wrote: > > I know the idea is to have very cheap boxes in clusters, but I wonder > > how much they're paying in shipping for replacing the cheap hardware. > Never underestimate the amount of airbills that can be paid with KISS > strategy.
Yep, that's true. Shipping is cheap, it's customs that's expensive and time-consuming, and Akamai tends to avoid the kind of places where you have to deal with a lot of customs.
I'd note that the original poster didn't classify 'broken' or 'outage' or 'non-functioning'... just the end result: "replacement".
So, is akamai doing some fancy SMART detection and seeing bad fans and replacing, seeing a bad cpu fan or disk or memory corruption and replacing, or are these hard box outages with no recourse but a complete immediate replacement?
As far as I can tell the only thing that will get a box replaced is if it can't be booted/pinged. We've pointed out dead CPU fans before (even on the incoming replacement boxes) and they've never seemed to care. If it runs it runs. If it doesn't they replace the entire box. Given all their redundancy I suppose that is probably the way to go. Chris -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Chris Owen ~ Garden City (620) 275-1900 ~ Lottery (noun): President ~ Wichita (316) 858-3000 ~ A stupidity tax Hubris Communications Inc ~ www.hubris.net ~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
On Mon, 28 Nov 2005, Chris Owen wrote:
As far as I can tell the only thing that will get a box replaced is if it can't be booted/pinged. We've pointed out dead CPU fans before (even on the incoming replacement boxes) and they've never seemed to care. If it runs it runs. If it doesn't they replace the entire box.
Having built a fair number of machines to live for 5 years or longer in data-centers I will never visit, there's relatively little that you want to triage onsite on a rackmount pc. Drives, in hot-plug enclosures and removable power supply modules are about it... Smart-hands are good for racking and stacking, swapping disks, recabling the oob, swapping media and so forth. It's not really a good use of someone else's time to have them performing experimental surgery on pc's. Much better to simply ship out another one and ship the old one back in the same box. Decent modern 1u chassis still have sufficient airflow with a couple fans failed to remain adequately cool, further there's now enough sensors in a pc to be able to tell when you getting in trouble, rpm indicator for all the fans, intake processor and output temperature, thermal sensors in each of the drives etc. Our success-rate at indetifying machines before they fail has gotten substantially better over time.
Given all their redundancy I suppose that is probably the way to go.
Chris
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Chris Owen ~ Garden City (620) 275-1900 ~ Lottery (noun): President ~ Wichita (316) 858-3000 ~ A stupidity tax Hubris Communications Inc ~ www.hubris.net ~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- -------------------------------------------------------------------------- Joel Jaeggli Unix Consulting joelja@darkwing.uoregon.edu GPG Key Fingerprint: 5C6E 0104 BAF0 40B0 5BD3 C38B F000 35AB B67F 56B2
Thought folks might find this interesting http://www.newscientist.com/article.ns?id=dn8403 Viral Cure Could 'Immunise' The Internet, New Scientist Excerpts: A cure for computer viruses that spreads in a viral fashion could immunise the internet, even against pests that travel at lightning speed, a mathematical study reveals. Most conventional anti-virus programs use "signatures" to identify and block viruses. But experts must first analyse a virus before sending out the fix. This means that rapidly spreading viruses can cause widespread damage before being stopped. Source: Viral Cure Could 'Immunise' The Internet, Kurt Kleiner, NewScientist, 05/12/01
sgorman1@gmu.edu wrote:
Thought folks might find this interesting
http://www.newscientist.com/article.ns?id=dn8403
Viral Cure Could 'Immunise' The Internet, New Scientist
Excerpts: A cure for computer viruses that spreads in a viral fashion could immunise the internet, even against pests that travel at lightning speed, a mathematical study reveals.
Most conventional anti-virus programs use "signatures" to identify and block viruses. But experts must first analyse a virus before sending out the fix. This means that rapidly spreading viruses can cause widespread damage before being stopped.
Source: Viral Cure Could 'Immunise' The Internet, Kurt Kleiner, NewScientist, 05/12/01
Skynet becoming self-aware anyone? -- Andre
At 9:30 -0500 12/9/05, sgorman1@gmu.edu wrote:
Thought folks might find this interesting
http://www.newscientist.com/article.ns?id=dn8403
Viral Cure Could 'Immunise' The Internet, New Scientist
Excerpts: A cure for computer viruses that spreads in a viral fashion could immunise the internet, even against pests that travel at lightning speed, a mathematical study reveals.
Most conventional anti-virus programs use "signatures" to identify and block viruses. But experts must first analyse a virus before sending out the fix. This means that rapidly spreading viruses can cause widespread damage before being stopped.
Source: Viral Cure Could 'Immunise' The Internet, Kurt Kleiner, NewScientist, 05/12/01
This has been thought of many times. My spin around this was attached to DARPA's Active Network (http://www.darpa.mil/ato/programs/AN/). The eternal question in security is "what are you defending against?" Because of that, security will always have a strong reactionary element. I can't cite any, but I recall hearing some claims that viruses in the past were meant to fix problems or highlight in a benign way the presence of problems. It's been tried in real life, I don't see that a mathematical study is going to come up with a result that is more meaningful. -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Edward Lewis +1-571-434-5468 NeuStar 3 months to the next trip. I guess it's finally time to settle down and find a grocery store.
On Friday 09 Dec 2005 14:57, you wrote:
a mathematical study is going to come up with a result that is more meaningful.
The story is badly headlined. The study is saying how many "canaries" do we need to keep the Internet safe, or how big the immune system needs to be, not about a viral cure as such. It says to keep the infection rate of new malware down to a fraction of a percent we'd need about a million or so machines looking for malware (on some assumptions). Should convince someone that we need to change the assumptions. Hopefully the modelling says something about what factors it is sensitive to. It does however remind me of the comparison of engineering and science, a scientist will earn a living by taking a really difficult problem and spends many years solving it, an engineer earns a living by finding really difficult problems and side stepping them.
Keep in mind the study was done by physicists, who while brilliant, cannot be bothered with operational realities that prevent their equations from being elegant. Still an interesting hypothesis on how to leverage network structure to fight infections - this assumes you buy into the whole Internet is "scale free" argument to begin with. ----- Original Message ----- From: Simon Waters <simonw@zynet.net> Date: Friday, December 9, 2005 10:34 am Subject: Re: Viral Cure Could 'Immunise' The Internet
On Friday 09 Dec 2005 14:57, you wrote:
a mathematical study is going to come up with a result that is more meaningful.
The story is badly headlined.
The study is saying how many "canaries" do we need to keep the Internet safe, or how big the immune system needs to be, not about a viral cure as such.
It says to keep the infection rate of new malware down to a fraction of a percent we'd need about a million or so machines looking for malware (on some assumptions).
Should convince someone that we need to change the assumptions. Hopefully the modelling says something about what factors it is sensitive to.
It does however remind me of the comparison of engineering and science, a scientist will earn a living by taking a really difficult problem and spends many years solving it, an engineer earns a living by finding really difficult problems and side stepping them.
On Fri, Dec 09, 2005 at 10:46:50AM -0500, sgorman1@gmu.edu wrote:
Keep in mind the study was done by physicists, who while brilliant, cannot be bothered with operational realities that prevent their equations from being elegant.
The Form Factor of a Cow: Summary http://graphics.cs.uiuc.edu/~garland/CMU/cow-formfac.html "For evaluating thermal radiant exchange between a cow and her surroundings, the cow can be represented by an equivalent sphere..." Eric, trained theoretical physicist who is bothered daily by the operational realities of his network... </returns to dreaming of becoming the maytag man...>
On Fri, 9 Dec 2005 sgorman1@gmu.edu wrote:
Keep in mind the study was done by physicists, who while brilliant, cannot be bothered with operational realities that prevent their equations from being elegant.
Still an interesting hypothesis on how to leverage network structure to fight infections - this assumes you buy into the whole Internet is "scale free" argument to begin with.
And you buy the argument that computer viruses are dumb. Malware is not naturally occuring and is not random. Crime on the Internet is becoming more lumpy.
sgorman1@gmu.edu wrote:
Thought folks might find this interesting
http://www.newscientist.com/article.ns?id=dn8403
Viral Cure Could 'Immunise' The Internet, New Scientist
Excerpts: A cure for computer viruses that spreads in a viral fashion could immunise the internet, even against pests that travel at lightning speed, a mathematical study reveals.
Most conventional anti-virus programs use "signatures" to identify and block viruses. But experts must first analyse a virus before sending out the fix. This means that rapidly spreading viruses can cause widespread damage before being stopped.
Source: Viral Cure Could 'Immunise' The Internet, Kurt Kleiner, NewScientist, 05/12/01
Sounds like: "I make your computer part of my botnet - only to prevent you from becomming part of somebodyelses botnet." How do I discriminate a real virus from a preventive one? I mean, how do I forge my virus so that you believe it is a preventive one? How about biology? AIDS works by attacking the immune system. If we had no white blood vessels there would be no AIDS. Some vermin does already use Anti Virus Systems to spread. Ok, if they use their preventive virus to kill all windows out there and replace it with a linux? Yes, that might be an idea. That would really stop the virus. ;) -- Peter and Karin Dambier The Public-Root Consortium Graeffstrasse 14 D-64646 Heppenheim +49(6252)671-788 (Telekom) +49(179)108-3978 (O2 Genion) +49(6252)750-308 (VoIP: sipgate.de) mail: peter@peter-dambier.de mail: peter@echnaton.serveftp.com http://iason.site.voila.fr
On Fri, 9 Dec 2005 sgorman1@gmu.edu wrote:
Thought folks might find this interesting
http://www.newscientist.com/article.ns?id=dn8403
Viral Cure Could 'Immunise' The Internet, New Scientist
Excerpts: A cure for computer viruses that spreads in a viral fashion could immunise the internet, even against pests that travel at lightning speed, a mathematical study reveals.
I think they call this 'welchia'... I don't recall it doing all that many 'good' things, unless 'raising customer traffic rates' qualifies as 'good'.
In message <Pine.GSO.4.58.0512091602160.20032@marvin.argfrp.us.uu.net>, "Christ opher L. Morrow" writes:
On Fri, 9 Dec 2005 sgorman1@gmu.edu wrote:
Thought folks might find this interesting
http://www.newscientist.com/article.ns?id=dn8403
Viral Cure Could 'Immunise' The Internet, New Scientist
Excerpts: A cure for computer viruses that spreads in a viral fashion could immunise the internet, even against pests that travel at lightning speed, a mathematical study reveals.
I think they call this 'welchia'... I don't recall it doing all that many 'good' things, unless 'raising customer traffic rates' qualifies as 'good'.
Besides, it's *unauthorized*. I don't want random strangers deciding when my machine should be patched. I think, in this forum, we all know that patches can have bad side effects. --Steven M. Bellovin, http://www.cs.columbia.edu/~smb
On Fri, 9 Dec 2005 sgorman1@gmu.edu wrote:
Excerpts: A cure for computer viruses that spreads in a viral fashion ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ could immunise the internet, even against pests that travel at lightning speed, a mathematical study reveals.
Nobody remembers Nachi/Welchia and the damage it did to networks while curing blaster? Nice intention. Bad idea. ---------------------------------------------------------------------- Jon Lewis | I route Senior Network Engineer | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
On Mon, 28 Nov 2005, Christian Kuhtz wrote:
Never underestimate the amount of airbills that can be paid with KISS strategy.
Especially since Akamai doesn't pay for truck rolls and man hours to get the replacements done onsite. -- Mikael Abrahamsson email: swmike@swm.pp.se
Never underestimate the amount of airbills that can be paid with KISS strategy.
Especially since Akamai doesn't pay for truck rolls and man hours to get the replacements done onsite.
I'm sorry, isn't that exactly what an airbill *is* paying for -- to get the equipment on site? The man hours (really, we are talking about less than a single hour to replace a server including all the mounting and repacking). The one man hour that they need (no more than 6 a year by the look of it) should offset the value the ISP is getting from not buying bandwidth to get to the content and for the improved performance they get. If that model doesn't work for the ISP in question, they should ask Akamai to pull their gear. DJ
On Mon, 28 Nov 2005, Deepak Jain wrote:
Never underestimate the amount of airbills that can be paid with KISS strategy.
Especially since Akamai doesn't pay for truck rolls and man hours to get the replacements done onsite.
I'm sorry, isn't that exactly what an airbill *is* paying for -- to get the equipment on site?
The man hours (really, we are talking about less than a single hour to replace a server including all the mounting and repacking). The one man hour that they need (no more than 6 a year by the look of it) should offset the value the ISP is getting from not buying bandwidth to get to the content and for the improved performance they get.
If that model doesn't work for the ISP in question, they should ask Akamai to pull their gear.
I didn't really get the impression that people were really complaining so much (I certainly wasn't) as they were just pointing out there was an issue. However, I do think Akamai would be better off getting their issues with their replacement boxes straightened out. I agree that we get value for having the boxes on our network (and so do they lets not forget). However, it is a bit frustrating to replace the same box 3 times in less than a month. Hauling a box down to the colo is no big deal but when the box you are taking down there has a dead CPU fan and two dead case fans it's hard not to think you might be wasting your time. It isn't just that they are wasting my time. They are also wasting their own time. It's the overall lack efficiency that bothers me ;-] Chris -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Chris Owen ~ Garden City (620) 275-1900 ~ Lottery (noun): President ~ Wichita (316) 858-3000 ~ A stupidity tax Hubris Communications Inc ~ www.hubris.net ~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Chris Owen wrote:
It isn't just that they are wasting my time. They are also wasting their own time. It's the overall lack efficiency that bothers me ;-]
Don't worry, it wont take long until google parks their datacenter-in-a-container outside at the fiber junction and the content distribution guys will be obsoleted overnight. Pete
It isn't just that they are wasting my time. They are also wasting their own time. It's the overall lack efficiency that bothers me ;-]
i suspect you have a datapoint on how they're doing financially. they ain't stoopid. they'll deal with it when the cost/benefit gets high enough on their priority list. isn't the first time that good s&m covers some technical gaps, and won't be the last. randy
In message <17291.29156.391354.250259@roam.psg.com>, Randy Bush writes:
It isn't just that they are wasting my time. They are also wasting their own time. It's the overall lack efficiency that bothers me ;-]
i suspect you have a datapoint on how they're doing financially. they ain't stoopid. they'll deal with it when the cost/benefit gets high enough on their priority list. isn't the first time that good s&m covers some technical gaps, and won't be the last.
To quote a science fiction story I'm fond of, "efficiency depends on what you want to effish". --Steven M. Bellovin, http://www.cs.columbia.edu/~smb
To quote a science fiction story I'm fond of, "efficiency depends on what you want to effish".
--Steven M. Bellovin, http://www.cs.columbia.edu/~smb
Sci-fi injection! (marking another beer owed) Gadi.
CO> Date: Mon, 28 Nov 2005 14:57:58 -0600 (CST) CO> From: Chris Owen CO> However, I do think Akamai would be better off getting their issues with CO> their replacement boxes straightened out. I agree that we get value for CO> having the boxes on our network (and so do they lets not forget). *shrug* It's not that expensive to ship boxen back and forth, and I'd hazard a guess they have people who troubleshoot the dead en masse. If a dead box costs $50, the question becomes how much more would prolonging box death cost? CO> However, it is a bit frustrating to replace the same box 3 times in less Heh. Never had _that_ bad, personally. CO> than a month. Hauling a box down to the colo is no big deal but when the Depends. In Kansas, no. In $big_metro_area during rush hour... well, I've learned why people state "distance" in terms of hours. :-) CO> box you are taking down there has a dead CPU fan and two dead case fans CO> it's hard not to think you might be wasting your time. True. So if the CPU fan is dead, just say the box is plugged in; act surprised when doesn't ping. ;-) CO> It isn't just that they are wasting my time. They are also wasting their CO> own time. It's the overall lack efficiency that bothers me ;-] There are enough clue-challenged networks that I wouldn't want arbitrary people playing around with my gear. Shipping can be more efficient. Eddy -- Everquick Internet - http://www.everquick.net/ A division of Brotsman & Dreger, Inc. - http://www.brotsman.com/ Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 785 865 5885 Lawrence and [inter]national Phone: +1 316 794 8922 Wichita ________________________________________________________________________ DO NOT send mail to the following addresses: davidc@brics.com -*- jfconmaapaq@intc.net -*- sam@everquick.net Sending mail to spambait addresses is a great way to get blocked. Ditto for broken OOO autoresponders and foolish AV software backscatter.
Deepak Jain wrote:
If that model doesn't work for the ISP in question, they should ask Akamai to pull their gear.
And hopefully they'll (someday) send servers in my direction - is their "minimum criteria" creeping upwards at the same rate as overall Internet traffic did in the late 90s? pt
On Mon, 28 Nov 2005, Pete Templin wrote:
And hopefully they'll (someday) send servers in my direction - is their "minimum criteria" creeping upwards at the same rate as overall Internet traffic did in the late 90s?
The impression I got was they originally scattered their machines to everyone who had a network with a growth plan and bought them a beer. Some people even got/get paid to host them. After the .com crash they started being a bit more careful about who they gave them to and doing a bit more analysis as whether a new site was worth the trouble. One way to get a cluster might be to suggest that your will make better use of it than a nearby company with a cluster that is much smaller than you. I have heard of people trying this in Australia, no idea how well it works. I know people who were doing under 10Mb/s via their clusters, but they are in Aus/NZ so the threshold might be higher elsewhere. -- Simon J. Lyall | Very Busy | Web: http://www.darkmere.gen.nz/ "To stay awake all night adds a day to your life" - Stilgar | eMT.
On Mon, 28 Nov 2005, Deepak Jain wrote:
I'm sorry, isn't that exactly what an airbill *is* paying for -- to get the equipment on site?
They also frequently need boxes power cycled. It got to be so frequent that we "gave them" a remote reboot switch for all their gear and told them how to use it. They still kept emailing us for reboots until I finally used a contact at akamai to get the remote reboot info properly placed. We've had our share of failed boxes, DOA boxes, boxes with components literally falling out of them on arrival, etc. I suspect it's just a sign of the box building having been farmed out to the cheapest available source. When you're building boxes in really large volume, what's a few missing screws here there? :)
The man hours (really, we are talking about less than a single hour to replace a server including all the mounting and repacking). The one man hour that they need (no more than 6 a year by the look of it) should offset the value the ISP is getting from not buying bandwidth to get to the content and for the improved performance they get.
I wouldn't count on that. With bandwidth prices continually falling, and the ISP business changing (at least for us, dialup/DSL is dying, hosting is taking off, and now instead of having spare outbound capacity to sell to Akamai), we do more outbound than inbound, so the servers really don't save us anything except maybe a bit of latency.
If that model doesn't work for the ISP in question, they should ask Akamai to pull their gear.
Think of the man hours that'd take, ripping them out, boxing them up, etc. :) ---------------------------------------------------------------------- Jon Lewis | I route Senior Network Engineer | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
I still have the original server that started garlic.com in production after 11+ years so I know servers can last a long time. I don't understand why Akamai failure rates are so high
Applications which cause the disk to thrash will wear out disk drives much more quickly than non-thrashing applications. When I still ran USENET news servers back before cyclic file systems were used, I remember that their hard drives died frequently, often after less than a year of service, but those drives were thrashing 24 by 7. You can hear drives thrashing and feel it by touching the case. It is caused by almost completely random access resulting in almost constant head movement. It is cost effective to just thrash cheap drives and replace them when they die. --Michael Dillon
i see lots of hardware failures. one particular problem i have noted is just kind of a bad case layout. the network card sits in a little pci riser card and is also bolted to the back of the case. it likes to unseat itself over time. eventually the MB dies. i'm not sure if one has anything to do with the other but i suspect it does because if i pull the screws out of the network card and just let it sti in the pci riser it doesnt unseat itself and the box tends to live a bit longer. but yes. i have more failures of akamai boxes on my network then anything by a long shot. chalk it up to cheap hardware i guess. On Mon, 28 Nov 2005, Roy wrote:
Date: Mon, 28 Nov 2005 10:39:51 -0800 From: Roy <garlic@garlic.com> To: nanog@merit.edu Subject: Akamai server reliability
Hi,
Many moons ago, we got a set of Akamai servers. Over the years I think they replaced every one of them at least once. Last August we got a another set of servers due to a move and now two of those three servers have failed.
I still have the original server that started garlic.com in production after 11+ years so I know servers can last a long time. I don't understand why Akamai failure rates are so high
Is anyone else seeing high failure rates of Akamai servers at their facilities?
Roy
Ryan Dobrynski Hat-Swapping Gnome Choice Communications (\_/) (O.o) (> < ) this is Bunny. Copy Bunny into your signature to help him on his way to world domination.
participants (27)
-
Andre Oppermann -
Bill Woodcock -
Chris Owen -
Christian Kuhtz -
Christopher L. Morrow -
Deepak Jain -
Edward B. Dreger -
Edward Lewis -
Eric Gauthier -
Gadi Evron -
Joel Jaeggli -
Jon Lewis -
Michael.Dillon＠btradianz.com -
Mikael Abrahamsson -
Mike Tancsa -
Pete Templin -
Peter Dambier -
Petri Helenius -
Randy Bush -
Roy -
Ryan Dobrynski -
Sean Donelan -
sgorman1＠gmu.edu -
Simon Lyall -
Simon Waters -
Steven M. Bellovin -
Vinny Abello