RE: 365x24x7 (sleep patterns)
Suggestion; once on the 'night shift' stay put for at least three months... Sleep patterns take time to adjust. Jumping between day and night shifts will burn out even the most motivated employee. Mark Green
From: nanog-request@nanog.org Subject: NANOG Digest, Vol 39, Issue 46 To: nanog@nanog.org Date: Fri, 15 Apr 2011 15:52:19 +0000
Send NANOG mailing list submissions to nanog@nanog.org
To subscribe or unsubscribe via the World Wide Web, visit https://mailman.nanog.org/mailman/listinfo/nanog or, via email, send a message with subject or body 'help' to nanog-request@nanog.org
You can reach the person managing the list at nanog-owner@nanog.org
When replying, please edit your Subject line so it is more specific than "Re: Contents of NANOG digest..."
Today's Topics:
1. 365x24x7 (harbor235) 2. Re: New hijacks, and lots of them (Ronald F. Guilmette) 3. Re: 365x24x7 (Peter Hicks) 4. Re: 365x24x7 (Charles Mills) 5. RE: 365x24x7 (Greg Moore) 6. Re: 365x24x7 (Steve Clark) 7. Re: 365x24x7 (Marshall Eubanks) 8. Re: 365x24x7 (Alex Brooks) 9. RE: 365x24x7 (Sanders, Randall K) 10. Re: 365x24x7 (Tony Finch) 11. Re: 365x24x7 (harbor235) 12. Re: 365x24x7 (Paul Graydon)
----------------------------------------------------------------------
Message: 1 Date: Fri, 15 Apr 2011 09:14:05 -0400 From: harbor235 <harbor235@gmail.com> Subject: 365x24x7 To: NANOG list <nanog@nanog.org> Message-ID: <BANLkTimD9jYVzM_hv21y9LHzObR3RVG0YA@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1
If I were going to provide a 365x24x7 NOC, how many teams of personnel do I need to fully cover operations? I assume minimally you need 3 teams to cover the required 24 hr coverage, but there is off time and schedule rotation?
thoughts, experience?
Mike
------------------------------
Message: 2 Date: Fri, 15 Apr 2011 06:17:06 -0700 From: "Ronald F. Guilmette" <rfg@tristatelogic.com> Subject: Re: New hijacks, and lots of them To: nanog@nanog.org Message-ID: <2971.1302873426@tristatelogic.com>
In message <5824.1302780126@tristatelogic.com>, I wrote:
http://www.47-usc-230c2.org/20110414-snowshoe-1.txt http://www.47-usc-230c2.org/20110414-snowshoe-2.txt
My apologies to anyone and everyone who tried to get at these files. It seems that my provider may perhaps have recently developed some rather odd ideas about packet filtering for static broadband lines. Until I can get the problem worked out, the following alternative URLs ought to do instead:
ftp://ftp.47-usc-230c2.org/pub/20110414-snowshoe-1.txt ftp://ftp.47-usc-230c2.org/pub/20110414-snowshoe-2.txt
------------------------------
Message: 3 Date: Fri, 15 Apr 2011 14:17:48 +0100 From: Peter Hicks <peter.hicks@poggs.co.uk> Subject: Re: 365x24x7 To: harbor235 <harbor235@gmail.com> Cc: NANOG list <nanog@nanog.org> Message-ID: <066F653E-C0F8-4026-9FE0-3A8899378D0C@poggs.co.uk> Content-Type: text/plain; charset=us-ascii
On 15 Apr 2011, at 14:14, harbor235 wrote:
If I were going to provide a 365x24x7 NOC, how many teams of personnel do I need to fully cover operations? I assume minimally you need 3 teams to cover the required 24 hr coverage, but there is off time and schedule rotation?
Although more geared up for on-call, http://blog.hinterlands.org/2010/07/running-an-oncall-rota/ is a very useful resource.
Peter
------------------------------
Message: 4 Date: Fri, 15 Apr 2011 09:25:02 -0400 From: Charles Mills <w3yni1@gmail.com> Subject: Re: 365x24x7 To: harbor235 <harbor235@gmail.com>, NANOG list <nanog@nanog.org> Message-ID: <BANLkTint8hnfenSQZBHoS3x4hAS_P78JNA@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1
I've had it done in places where I work where you'll have 3 rotations working 12 hour shifts.
In a 2 week pay period they get their 80 hours in a blend 36 one week and 44 the next. It gives some nice consecutive days off time which also doubles as a retention tool for some employees. You might have to get creative to have all the days work out but it can be done.
On Fri, Apr 15, 2011 at 9:14 AM, harbor235 <harbor235@gmail.com> wrote:
If I were going to provide a 365x24x7 NOC, how many teams of personnel do I need to fully cover operations? I assume minimally you need 3 teams to cover the required 24 hr coverage, but there is off time and schedule rotation?
thoughts, experience?
Mike
-- ===================================== Charles L. Mills Email: w3yni1@gmail.com ===================================== Need server hosting, DR or colocation services?? See me!
------------------------------
Message: 5 Date: Fri, 15 Apr 2011 09:37:54 -0400 From: Greg Moore <mooregr@greenms.com> Subject: RE: 365x24x7 To: harbor235 <harbor235@gmail.com>, NANOG list <nanog@nanog.org> Message-ID: <mailman.2806.1302882739.16627.nanog@nanog.org> Content-Type: text/plain; charset="iso-8859-1"
When I did this years ago I found 5 was really a minimum so that I could cover weekends and then had extra coverage as needed during the week.
I did find it was good to swap out the graveyard shift every 6 months or so.
-----Original Message----- From: harbor235 Sent: Friday, April 15, 2011 9:14 AM To: NANOG list Subject: 365x24x7
If I were going to provide a 365x24x7 NOC, how many teams of personnel do I need to fully cover operations? I assume minimally you need 3 teams to cover the required 24 hr coverage, but there is off time and schedule rotation?
thoughts, experience?
Mike
------------------------------
Message: 6 Date: Fri, 15 Apr 2011 09:48:42 -0400 From: Steve Clark <sclark@netwolves.com> Subject: Re: 365x24x7 To: Charles Mills <w3yni1@gmail.com> Cc: NANOG list <nanog@nanog.org> Message-ID: <4DA84CBA.5030409@netwolves.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
On 04/15/2011 09:25 AM, Charles Mills wrote:
I've had it done in places where I work where you'll have 3 rotations working 12 hour shifts.
In a 2 week pay period they get their 80 hours in a blend 36 one week and 44 the next. It gives some nice consecutive days off time which also doubles as a retention tool for some employees. You might have to get creative to have all the days work out but it can be done.
On Fri, Apr 15, 2011 at 9:14 AM, harbor235<harbor235@gmail.com> wrote:
If I were going to provide a 365x24x7 NOC, how many teams of personnel do I need to fully cover operations? I assume minimally you need 3 teams to cover the required 24 hr coverage, but there is off time and schedule rotation?
thoughts, experience?
Mike
I worked once in a production plant. They had 4 crews a,b,c,d and worked 6 days on 2 off. It was rotating shifts. As an example crew A worked 6 days 8-4 was off 2 days then started the next shift on 4-midnight, then midnight to 8. So at any one day 3 of the 4 crews were working and the other crew was off.
-- Stephen Clark *NetWolves* Sr. Software Engineer III Phone: 813-579-3200 Fax: 813-882-0209 Email: steve.clark@netwolves.com http://www.netwolves.com
------------------------------
Message: 7 Date: Fri, 15 Apr 2011 09:52:11 -0400 From: Marshall Eubanks <tme@americafree.tv> Subject: Re: 365x24x7 To: Greg Moore <mooregr@greenms.com> Cc: NANOG list <nanog@nanog.org> Message-ID: <CAC57439-9CCC-42E3-8CB4-FA4141D8F3DA@americafree.tv> Content-Type: text/plain; charset=us-ascii
On Apr 15, 2011, at 9:37 AM, Greg Moore wrote:
When I did this years ago I found 5 was really a minimum so that I could cover weekends and then had extra coverage as needed during the week.
I did find it was good to swap out the graveyard shift every 6 months or so.
When I worked with NASA and the Navy on remote locations that needed full time staffing, the rule of thumb was 5 people and 4 shifts was the absolute minimum, and the people had to be motivated enough to pull 12 hour shifts on a regular basis (i.e., this was very bare bones). The 4th shift was needed during the weekends.
Anything less, and you would have uncovered periods if, say, 2 people got sick simultaneously.
Regards Marshall
-----Original Message----- From: harbor235 Sent: Friday, April 15, 2011 9:14 AM To: NANOG list Subject: 365x24x7
If I were going to provide a 365x24x7 NOC, how many teams of personnel do I need to fully cover operations? I assume minimally you need 3 teams to cover the required 24 hr coverage, but there is off time and schedule rotation?
thoughts, experience?
Mike
------------------------------
Message: 8 Date: Fri, 15 Apr 2011 15:27:32 +0100 From: Alex Brooks <askoorb+nanog@gmail.com> Subject: Re: 365x24x7 To: nanog <nanog@nanog.org> Message-ID: <BANLkTikkVH_yk=97oWgq_A0YLxZ9dxNVXg@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1
On Fri, Apr 15, 2011 at 2:14 PM, harbor235 <harbor235@gmail.com> wrote:
If I were going to provide a 365x24x7 NOC, how many teams of personnel do I need to fully cover operations? I assume minimally you need 3 teams to cover the required 24 hr coverage, but there is off time and schedule rotation?
Well, if you feel like being "nice" to your employees, or want to stop them from making mistakes a few months/years in to shift work, (or if you're having to set something up abroad), the Working Time Directive can be a useful guideline.
(Full details at http://en.wikipedia.org/wiki/Working_Time_Directive and http://www.businesslink.gov.uk/bdotg/action/layer?topicId=1073858926 if you're board, ignore the silly bits)
But basically, in general, workers aged 18 and over are entitled to:
- work no more than six days out of every seven, or 12 out of every 14 - take a 20-minute break if their shift lasts for more than six hours - work a maximum 48-hour average week
And in general, night workers:
- should not work more than an average of eight hours in a 24-hour period, averaged over a reference period of 17 weeks
If you're an employer, be glad you're in North America :-)
HTH,
Alex
------------------------------
Message: 9 Date: Fri, 15 Apr 2011 09:31:48 -0500 From: "Sanders, Randall K" <Randall.K.Sanders@xo.com> Subject: RE: 365x24x7 To: nanog <nanog@nanog.org> Message-ID: <B0E45381691B4E4188ECEE2507142CFC5C7B3660@txplanexch01.corp.inthosts.net>
Content-Type: text/plain; charset="us-ascii"
Have you taken into account number of alarms per hour, inbound call volume for repairs, and how much repair is done at the first tier level? Bare minimum staffing in a busy environment?
Randy Sanders
-----Original Message----- From: Alex Brooks [mailto:askoorb+nanog@gmail.com] Sent: Friday, April 15, 2011 9:28 AM To: nanog Subject: Re: 365x24x7
On Fri, Apr 15, 2011 at 2:14 PM, harbor235 <harbor235@gmail.com> wrote:
If I were going to provide a 365x24x7 NOC, how many teams of personnel do I need to fully cover operations? I assume minimally you need 3 teams to cover the required 24 hr coverage, but there is off time and schedule rotation?
Well, if you feel like being "nice" to your employees, or want to stop them from making mistakes a few months/years in to shift work, (or if you're having to set something up abroad), the Working Time Directive can be a useful guideline.
(Full details at http://en.wikipedia.org/wiki/Working_Time_Directive and http://www.businesslink.gov.uk/bdotg/action/layer?topicId=1073858926 if you're board, ignore the silly bits)
But basically, in general, workers aged 18 and over are entitled to:
- work no more than six days out of every seven, or 12 out of every 14 - take a 20-minute break if their shift lasts for more than six hours - work a maximum 48-hour average week
And in general, night workers:
- should not work more than an average of eight hours in a 24-hour period, averaged over a reference period of 17 weeks
If you're an employer, be glad you're in North America :-)
HTH,
Alex
------------------------------
Message: 10 Date: Fri, 15 Apr 2011 15:36:35 +0100 From: Tony Finch <dot@dotat.at> Subject: Re: 365x24x7 To: harbor235 <harbor235@gmail.com> Cc: NANOG list <nanog@nanog.org> Message-ID: <alpine.LSU.2.00.1104151530010.19442@hermes-2.csi.cam.ac.uk> Content-Type: TEXT/PLAIN; charset=US-ASCII
harbor235 <harbor235@gmail.com> wrote:
If I were going to provide a 365x24x7 NOC, how many teams of personnel do I need to fully cover operations?
Hours in the working year = 8 * 5 * 48 = 1920 Hours in the calendar year = 24 * 7 * 52 = 8736 Ratio = 4.55
Tony. -- f.anthony.n.finch <dot@dotat.at> http://dotat.at/ Rockall, Malin, Hebrides: South 5 to 7, occasionally gale 8 at first in Rockall and Malin, veering west or northwest 4 or 5, then backing southwest 5 or 6 later. Rough or very rough. Occasional rain. Moderate or good, occasionally poor.
------------------------------
Message: 11 Date: Fri, 15 Apr 2011 10:52:02 -0400 From: harbor235 <harbor235@gmail.com> Subject: Re: 365x24x7 To: NANOG list <nanog@nanog.org> Message-ID: <BANLkTinLSBedu3hq8hg4ihjEMcOHHbRjGg@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1
Guys,
Thanx alot, there is some great stuff here, also some stuff I had not thought of.
thanx again,
Mike
On Fri, Apr 15, 2011 at 10:36 AM, Tony Finch <dot@dotat.at> wrote:
harbor235 <harbor235@gmail.com> wrote:
If I were going to provide a 365x24x7 NOC, how many teams of personnel do I need to fully cover operations?
Hours in the working year = 8 * 5 * 48 = 1920 Hours in the calendar year = 24 * 7 * 52 = 8736 Ratio = 4.55
Tony. -- f.anthony.n.finch <dot@dotat.at> http://dotat.at/ Rockall, Malin, Hebrides: South 5 to 7, occasionally gale 8 at first in Rockall and Malin, veering west or northwest 4 or 5, then backing southwest 5 or 6 later. Rough or very rough. Occasional rain. Moderate or good, occasionally poor.
------------------------------
Message: 12 Date: Fri, 15 Apr 2011 05:52:13 -1000 From: Paul Graydon <paul@paulgraydon.co.uk> Subject: Re: 365x24x7 To: nanog@nanog.org Message-ID: <4DA869AD.9070202@paulgraydon.co.uk> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
On 4/15/2011 3:14 AM, harbor235 wrote:
If I were going to provide a 365x24x7 NOC, how many teams of personnel do I need to fully cover operations? I assume minimally you need 3 teams to cover the required 24 hr coverage, but there is off time and schedule rotation?
thoughts, experience?
Mike
For what it's worth, was part of a datacenter operations department that had a 24x7 team. 4 shifts, 4 staff on each shift (1 was supervisor who did same work as the rest, 1 'point of contact' who stayed in the office). 4 days on, 4 days off, 12 hour shifts, 8-8. Shift teams would alternate between day and night (so 4 day, 4 off, 4 night, 4 off, repeat ad infinitum). During the day that was bolstered by 6 day-staff, Monday to Friday, who would have a staggered start through the day (IIRC 2 start at 8, 2 at 9, 2 at 11)
------------------------------
_______________________________________________ NANOG mailing list NANOG@nanog.org https://mailman.nanog.org/mailman/listinfo/nanog
End of NANOG Digest, Vol 39, Issue 46 *************************************
+1. I'd go to six months, having been the night shift bitch. Flipping shifts around damn near killed me. On Fri, Apr 15, 2011 at 11:44 AM, Mark Green <ktm200exc@hotmail.com> wrote:
Suggestion; once on the 'night shift' stay put for at least three months... Sleep patterns take time to adjust. Jumping between day and night shifts will burn out even the most motivated employee.
Mark Green
..snip..
On Fri, Apr 15, 2011 at 11:53:47AM -0500, Chad Dailey wrote:
+1. I'd go to six months, having been the night shift bitch. Flipping shifts around damn near killed me.
On Fri, Apr 15, 2011 at 11:44 AM, Mark Green <ktm200exc@hotmail.com> wrote:
Suggestion; once on the 'night shift' stay put for at least three months... Sleep patterns take time to adjust. Jumping between day and night shifts will burn out even the most motivated employee.
Amen. There is evidence that, other things being relatively equal, people working rotating shifts have shorter life expectancies and that the faster the rotation, the shorter the expectancy gets. There also is some evidence that people working rotating shifts are more likely to get cancer. My experience: 6 on, 2 off, 8 hours, rotating to the next later shift: I never, ever got enough sleep -- for 2 years. 6 on, 2 off, 12 hours, straight mids, no rotation: much less bad. 5 on, 2 off, 8 hours, straight mids: quite tolerable. 5 on, 2 off, 8 hours, straight swings (1600-0000): out of phase with the world. YMMV; I expect it to. -- Mike Andrews, W5EGO mikea@mikea.ath.cx Tired old sysadmin
On 4/15/2011 10:11 AM, mikea wrote:
My experience:
6 on, 2 off, 8 hours, rotating to the next later shift: I never, ever got enough sleep -- for 2 years.
6 on, 2 off, 12 hours, straight mids, no rotation: much less bad.
5 on, 2 off, 8 hours, straight mids: quite tolerable.
5 on, 2 off, 8 hours, straight swings (1600-0000): out of phase with the world.
I've done all of the above but the 12 hour shift and can add 5 on, 2 off, 8 hours, rotating between swings and mids. They sucked. I'm in general agreement with Mike's judgments as well. If you want to be fair to your people and help keep their morale up, straight shifts is the way to go - or at least fix the mids shift and make the swing/mids switch at 2200 (10pm). Changing sleep times is the quickest way to get zombies for employees. If you try to do a 6 and 2 style rotation, eventually some smart person is going to figure out that they're getting screwed out of a lot of weekend and holiday time as opposed to the "daybeggers." My recommendation, based on 10 years of this nonsense in the Army, is minimum 2 people per shift, 5 on, 2 off, stagger the weekends so that someone gets Fri-Sat, the other Sun-Mon. If they can decide which wants which weekend between themselves, so much the better. As the FAA has lately demonstrated, single person night shifts is generally a bad thing if you actually want them to stay awake. -- Jeff Shultz
As the FAA has lately demonstrated, single person night shifts is generally a bad thing if you actually want them to stay awake.
-- Jeff Shultz
Jeff, there are other reasons for not having a single individual on an overnight shift. A person can have an unexpected medical emergency at any time or experience an accident. Getting help to them quickly can make a difference between life and death. Even choking on a meal can have a completely different outcome if there is another person available. Having only one person in a facility overnight by themselves just isn't a good idea, in my opinion.
On Fri, 15 Apr 2011, Jeff Shultz wrote:
On 4/15/2011 10:11 AM, mikea wrote:
My experience:
6 on, 2 off, 8 hours, rotating to the next later shift: I never, ever got enough sleep -- for 2 years.
6 on, 2 off, 12 hours, straight mids, no rotation: much less bad.
5 on, 2 off, 8 hours, straight mids: quite tolerable.
5 on, 2 off, 8 hours, straight swings (1600-0000): out of phase with the world.
I've done all of the above but the 12 hour shift and can add 5 on, 2 off, 8 hours, rotating between swings and mids. They sucked. I'm in general agreement with Mike's judgments as well.
If you want to be fair to your people and help keep their morale up, straight shifts is the way to go - or at least fix the mids shift and make the swing/mids switch at 2200 (10pm). Changing sleep times is the quickest way to get zombies for employees.
Local emergency services[1] operate '2 days, 2 nights, 4 off'. Dayshifts are 10 hour 8am-6pm. Nightshift is 6pm until 8am. This creates a 4-watch rotation. The day shifts are relatively normal working hours, and the 4 consecutive days off are handy, and as it's an 8 day cycle it slowly rotates so that you can wind up with weekends or weekdays off over time (which in itself can be handy). The guys employed this way are often given a little handbook with their year printed on it so they can plan their lives. Mark. [1] Fire and Ambulance services operate this way to my knowledge, Police is different depending on the part of the country you're in...
----- Original Message -----
From: "Mark Foster" <blakjak@blakjak.net>
Local emergency services[1] operate '2 days, 2 nights, 4 off'.
Dayshifts are 10 hour 8am-6pm. Nightshift is 6pm until 8am. This creates a 4-watch rotation.
I dunno from Ambulance -- they're load driven... by my understanding is that around here, the fire people are 3 days on, and 4 days off, or something similar to that. Since they sleep in, they're effectively on-call at all times, and they've got enough people on a shift that they can do internal rotations as to who goes, unless it's a big enough call that they all need to roll, which is a small enough percentage of calls to make it work. Cheers, - jra
In a past work life, there was a short experimental run where it was believed that the company I worked for could achieve 24/7 coverage through individuals being on-call throughout the entire weekend AND doing overnight maintenance during the week in 12 hour daily shifts from 8PM to 8AM. Needless to say, coming from a daytime schedule one week, covering all pages on a weekend which prevented you from getting much sleep, working 5 12 hour shifts in the following week on shortened sleep cycles (over 100 hours in total with the on-call and 12 hour shifts), then switching back to daytime hours the next week took a toll on me rather quickly. I think we got an extra week day off in there somewhere to recover the following week, but it was basically like running a person into the ground until they were almost dead, then letting them recover while the same thing was done to the next person. There were only 4 people to abuse like this at the time so it happened once a month. Luckily everyone came to their senses and realized this wasn't sustainable and it didn't last for more than a few months total. Moral of the story, don't do this to people unless you're into torture. :) -Vinny On Fri, 15 Apr 2011 11:53:47 -0500, Chad Dailey wrote:
+1. I'd go to six months, having been the night shift bitch. Flipping shifts around damn near killed me.
On Fri, Apr 15, 2011 at 11:44 AM, Mark Green <ktm200exc@hotmail.com> wrote:
Suggestion; once on the 'night shift' stay put for at least three months... Sleep patterns take time to adjust. Jumping between day and night shifts will burn out even the most motivated employee.
Mark Green
..snip..
On Apr 15, 2011, at 12:44 PM, Mark Green wrote:
Suggestion; once on the 'night shift' stay put for at least three months... Sleep patterns take time to adjust. Jumping between day and night shifts will burn out even the most motivated employee.
What we found was that we would find people who wanted to be on the night shift, and would NOT like to be changed, at all. Some people like night work, or have family situations where it is ideal for them. Regards Marshall
Mark Green
From: nanog-request@nanog.org Subject: NANOG Digest, Vol 39, Issue 46 To: nanog@nanog.org Date: Fri, 15 Apr 2011 15:52:19 +0000
Send NANOG mailing list submissions to nanog@nanog.org
To subscribe or unsubscribe via the World Wide Web, visit https://mailman.nanog.org/mailman/listinfo/nanog or, via email, send a message with subject or body 'help' to nanog-request@nanog.org
You can reach the person managing the list at nanog-owner@nanog.org
When replying, please edit your Subject line so it is more specific than "Re: Contents of NANOG digest..."
Today's Topics:
1. 365x24x7 (harbor235) 2. Re: New hijacks, and lots of them (Ronald F. Guilmette) 3. Re: 365x24x7 (Peter Hicks) 4. Re: 365x24x7 (Charles Mills) 5. RE: 365x24x7 (Greg Moore) 6. Re: 365x24x7 (Steve Clark) 7. Re: 365x24x7 (Marshall Eubanks) 8. Re: 365x24x7 (Alex Brooks) 9. RE: 365x24x7 (Sanders, Randall K) 10. Re: 365x24x7 (Tony Finch) 11. Re: 365x24x7 (harbor235) 12. Re: 365x24x7 (Paul Graydon)
----------------------------------------------------------------------
Message: 1 Date: Fri, 15 Apr 2011 09:14:05 -0400 From: harbor235 <harbor235@gmail.com> Subject: 365x24x7 To: NANOG list <nanog@nanog.org> Message-ID: <BANLkTimD9jYVzM_hv21y9LHzObR3RVG0YA@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1
If I were going to provide a 365x24x7 NOC, how many teams of personnel do I need to fully cover operations? I assume minimally you need 3 teams to cover the required 24 hr coverage, but there is off time and schedule rotation?
thoughts, experience?
Mike
------------------------------
Message: 2 Date: Fri, 15 Apr 2011 06:17:06 -0700 From: "Ronald F. Guilmette" <rfg@tristatelogic.com> Subject: Re: New hijacks, and lots of them To: nanog@nanog.org Message-ID: <2971.1302873426@tristatelogic.com>
In message <5824.1302780126@tristatelogic.com>, I wrote:
http://www.47-usc-230c2.org/20110414-snowshoe-1.txt http://www.47-usc-230c2.org/20110414-snowshoe-2.txt
My apologies to anyone and everyone who tried to get at these files. It seems that my provider may perhaps have recently developed some rather odd ideas about packet filtering for static broadband lines. Until I can get the problem worked out, the following alternative URLs ought to do instead:
ftp://ftp.47-usc-230c2.org/pub/20110414-snowshoe-1.txt ftp://ftp.47-usc-230c2.org/pub/20110414-snowshoe-2.txt
------------------------------
Message: 3 Date: Fri, 15 Apr 2011 14:17:48 +0100 From: Peter Hicks <peter.hicks@poggs.co.uk> Subject: Re: 365x24x7 To: harbor235 <harbor235@gmail.com> Cc: NANOG list <nanog@nanog.org> Message-ID: <066F653E-C0F8-4026-9FE0-3A8899378D0C@poggs.co.uk> Content-Type: text/plain; charset=us-ascii
On 15 Apr 2011, at 14:14, harbor235 wrote:
If I were going to provide a 365x24x7 NOC, how many teams of personnel do I need to fully cover operations? I assume minimally you need 3 teams to cover the required 24 hr coverage, but there is off time and schedule rotation?
Although more geared up for on-call, http://blog.hinterlands.org/2010/07/running-an-oncall-rota/ is a very useful resource.
Peter
------------------------------
Message: 4 Date: Fri, 15 Apr 2011 09:25:02 -0400 From: Charles Mills <w3yni1@gmail.com> Subject: Re: 365x24x7 To: harbor235 <harbor235@gmail.com>, NANOG list <nanog@nanog.org> Message-ID: <BANLkTint8hnfenSQZBHoS3x4hAS_P78JNA@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1
I've had it done in places where I work where you'll have 3 rotations working 12 hour shifts.
In a 2 week pay period they get their 80 hours in a blend 36 one week and 44 the next. It gives some nice consecutive days off time which also doubles as a retention tool for some employees. You might have to get creative to have all the days work out but it can be done.
On Fri, Apr 15, 2011 at 9:14 AM, harbor235 <harbor235@gmail.com> wrote:
If I were going to provide a 365x24x7 NOC, how many teams of personnel do I need to fully cover operations? I assume minimally you need 3 teams to cover the required 24 hr coverage, but there is off time and schedule rotation?
thoughts, experience?
Mike
-- ===================================== Charles L. Mills Email: w3yni1@gmail.com ===================================== Need server hosting, DR or colocation services?? See me!
------------------------------
Message: 5 Date: Fri, 15 Apr 2011 09:37:54 -0400 From: Greg Moore <mooregr@greenms.com> Subject: RE: 365x24x7 To: harbor235 <harbor235@gmail.com>, NANOG list <nanog@nanog.org> Message-ID: <mailman.2806.1302882739.16627.nanog@nanog.org> Content-Type: text/plain; charset="iso-8859-1"
When I did this years ago I found 5 was really a minimum so that I could cover weekends and then had extra coverage as needed during the week.
I did find it was good to swap out the graveyard shift every 6 months or so.
-----Original Message----- From: harbor235 Sent: Friday, April 15, 2011 9:14 AM To: NANOG list Subject: 365x24x7
If I were going to provide a 365x24x7 NOC, how many teams of personnel do I need to fully cover operations? I assume minimally you need 3 teams to cover the required 24 hr coverage, but there is off time and schedule rotation?
thoughts, experience?
Mike
------------------------------
Message: 6 Date: Fri, 15 Apr 2011 09:48:42 -0400 From: Steve Clark <sclark@netwolves.com> Subject: Re: 365x24x7 To: Charles Mills <w3yni1@gmail.com> Cc: NANOG list <nanog@nanog.org> Message-ID: <4DA84CBA.5030409@netwolves.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
On 04/15/2011 09:25 AM, Charles Mills wrote:
I've had it done in places where I work where you'll have 3 rotations working 12 hour shifts.
In a 2 week pay period they get their 80 hours in a blend 36 one week and 44 the next. It gives some nice consecutive days off time which also doubles as a retention tool for some employees. You might have to get creative to have all the days work out but it can be done.
On Fri, Apr 15, 2011 at 9:14 AM, harbor235<harbor235@gmail.com> wrote:
If I were going to provide a 365x24x7 NOC, how many teams of personnel do I need to fully cover operations? I assume minimally you need 3 teams to cover the required 24 hr coverage, but there is off time and schedule rotation?
thoughts, experience?
Mike
I worked once in a production plant. They had 4 crews a,b,c,d and worked 6 days on 2 off. It was rotating shifts. As an example crew A worked 6 days 8-4 was off 2 days then started the next shift on 4-midnight, then midnight to 8. So at any one day 3 of the 4 crews were working and the other crew was off.
-- Stephen Clark *NetWolves* Sr. Software Engineer III Phone: 813-579-3200 Fax: 813-882-0209 Email: steve.clark@netwolves.com http://www.netwolves.com
------------------------------
Message: 7 Date: Fri, 15 Apr 2011 09:52:11 -0400 From: Marshall Eubanks <tme@americafree.tv> Subject: Re: 365x24x7 To: Greg Moore <mooregr@greenms.com> Cc: NANOG list <nanog@nanog.org> Message-ID: <CAC57439-9CCC-42E3-8CB4-FA4141D8F3DA@americafree.tv> Content-Type: text/plain; charset=us-ascii
On Apr 15, 2011, at 9:37 AM, Greg Moore wrote:
When I did this years ago I found 5 was really a minimum so that I could cover weekends and then had extra coverage as needed during the week.
I did find it was good to swap out the graveyard shift every 6 months or so.
When I worked with NASA and the Navy on remote locations that needed full time staffing, the rule of thumb was 5 people and 4 shifts was the absolute minimum, and the people had to be motivated enough to pull 12 hour shifts on a regular basis (i.e., this was very bare bones). The 4th shift was needed during the weekends.
Anything less, and you would have uncovered periods if, say, 2 people got sick simultaneously.
Regards Marshall
-----Original Message----- From: harbor235 Sent: Friday, April 15, 2011 9:14 AM To: NANOG list Subject: 365x24x7
If I were going to provide a 365x24x7 NOC, how many teams of personnel do I need to fully cover operations? I assume minimally you need 3 teams to cover the required 24 hr coverage, but there is off time and schedule rotation?
thoughts, experience?
Mike
------------------------------
Message: 8 Date: Fri, 15 Apr 2011 15:27:32 +0100 From: Alex Brooks <askoorb+nanog@gmail.com> Subject: Re: 365x24x7 To: nanog <nanog@nanog.org> Message-ID: <BANLkTikkVH_yk=97oWgq_A0YLxZ9dxNVXg@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1
On Fri, Apr 15, 2011 at 2:14 PM, harbor235 <harbor235@gmail.com> wrote:
If I were going to provide a 365x24x7 NOC, how many teams of personnel do I need to fully cover operations? I assume minimally you need 3 teams to cover the required 24 hr coverage, but there is off time and schedule rotation?
Well, if you feel like being "nice" to your employees, or want to stop them from making mistakes a few months/years in to shift work, (or if you're having to set something up abroad), the Working Time Directive can be a useful guideline.
(Full details at http://en.wikipedia.org/wiki/Working_Time_Directive and http://www.businesslink.gov.uk/bdotg/action/layer?topicId=1073858926 if you're board, ignore the silly bits)
But basically, in general, workers aged 18 and over are entitled to:
- work no more than six days out of every seven, or 12 out of every 14 - take a 20-minute break if their shift lasts for more than six hours - work a maximum 48-hour average week
And in general, night workers:
- should not work more than an average of eight hours in a 24-hour period, averaged over a reference period of 17 weeks
If you're an employer, be glad you're in North America :-)
HTH,
Alex
------------------------------
Message: 9 Date: Fri, 15 Apr 2011 09:31:48 -0500 From: "Sanders, Randall K" <Randall.K.Sanders@xo.com> Subject: RE: 365x24x7 To: nanog <nanog@nanog.org> Message-ID: <B0E45381691B4E4188ECEE2507142CFC5C7B3660@txplanexch01.corp.inthosts.net>
Content-Type: text/plain; charset="us-ascii"
Have you taken into account number of alarms per hour, inbound call volume for repairs, and how much repair is done at the first tier level? Bare minimum staffing in a busy environment?
Randy Sanders
-----Original Message----- From: Alex Brooks [mailto:askoorb+nanog@gmail.com] Sent: Friday, April 15, 2011 9:28 AM To: nanog Subject: Re: 365x24x7
On Fri, Apr 15, 2011 at 2:14 PM, harbor235 <harbor235@gmail.com> wrote:
If I were going to provide a 365x24x7 NOC, how many teams of personnel do I need to fully cover operations? I assume minimally you need 3 teams to cover the required 24 hr coverage, but there is off time and schedule rotation?
Well, if you feel like being "nice" to your employees, or want to stop them from making mistakes a few months/years in to shift work, (or if you're having to set something up abroad), the Working Time Directive can be a useful guideline.
(Full details at http://en.wikipedia.org/wiki/Working_Time_Directive and http://www.businesslink.gov.uk/bdotg/action/layer?topicId=1073858926 if you're board, ignore the silly bits)
But basically, in general, workers aged 18 and over are entitled to:
- work no more than six days out of every seven, or 12 out of every 14 - take a 20-minute break if their shift lasts for more than six hours - work a maximum 48-hour average week
And in general, night workers:
- should not work more than an average of eight hours in a 24-hour period, averaged over a reference period of 17 weeks
If you're an employer, be glad you're in North America :-)
HTH,
Alex
------------------------------
Message: 10 Date: Fri, 15 Apr 2011 15:36:35 +0100 From: Tony Finch <dot@dotat.at> Subject: Re: 365x24x7 To: harbor235 <harbor235@gmail.com> Cc: NANOG list <nanog@nanog.org> Message-ID: <alpine.LSU.2.00.1104151530010.19442@hermes-2.csi.cam.ac.uk> Content-Type: TEXT/PLAIN; charset=US-ASCII
harbor235 <harbor235@gmail.com> wrote:
If I were going to provide a 365x24x7 NOC, how many teams of personnel do I need to fully cover operations?
Hours in the working year = 8 * 5 * 48 = 1920 Hours in the calendar year = 24 * 7 * 52 = 8736 Ratio = 4.55
Tony. -- f.anthony.n.finch <dot@dotat.at> http://dotat.at/ Rockall, Malin, Hebrides: South 5 to 7, occasionally gale 8 at first in Rockall and Malin, veering west or northwest 4 or 5, then backing southwest 5 or 6 later. Rough or very rough. Occasional rain. Moderate or good, occasionally poor.
------------------------------
Message: 11 Date: Fri, 15 Apr 2011 10:52:02 -0400 From: harbor235 <harbor235@gmail.com> Subject: Re: 365x24x7 To: NANOG list <nanog@nanog.org> Message-ID: <BANLkTinLSBedu3hq8hg4ihjEMcOHHbRjGg@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1
Guys,
Thanx alot, there is some great stuff here, also some stuff I had not thought of.
thanx again,
Mike
On Fri, Apr 15, 2011 at 10:36 AM, Tony Finch <dot@dotat.at> wrote:
harbor235 <harbor235@gmail.com> wrote:
If I were going to provide a 365x24x7 NOC, how many teams of personnel do I need to fully cover operations?
Hours in the working year = 8 * 5 * 48 = 1920 Hours in the calendar year = 24 * 7 * 52 = 8736 Ratio = 4.55
Tony. -- f.anthony.n.finch <dot@dotat.at> http://dotat.at/ Rockall, Malin, Hebrides: South 5 to 7, occasionally gale 8 at first in Rockall and Malin, veering west or northwest 4 or 5, then backing southwest 5 or 6 later. Rough or very rough. Occasional rain. Moderate or good, occasionally poor.
------------------------------
Message: 12 Date: Fri, 15 Apr 2011 05:52:13 -1000 From: Paul Graydon <paul@paulgraydon.co.uk> Subject: Re: 365x24x7 To: nanog@nanog.org Message-ID: <4DA869AD.9070202@paulgraydon.co.uk> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
On 4/15/2011 3:14 AM, harbor235 wrote:
If I were going to provide a 365x24x7 NOC, how many teams of personnel do I need to fully cover operations? I assume minimally you need 3 teams to cover the required 24 hr coverage, but there is off time and schedule rotation?
thoughts, experience?
Mike
For what it's worth, was part of a datacenter operations department that had a 24x7 team. 4 shifts, 4 staff on each shift (1 was supervisor who did same work as the rest, 1 'point of contact' who stayed in the office). 4 days on, 4 days off, 12 hour shifts, 8-8. Shift teams would alternate between day and night (so 4 day, 4 off, 4 night, 4 off, repeat ad infinitum). During the day that was bolstered by 6 day-staff, Monday to Friday, who would have a staggered start through the day (IIRC 2 start at 8, 2 at 9, 2 at 11)
------------------------------
_______________________________________________ NANOG mailing list NANOG@nanog.org https://mailman.nanog.org/mailman/listinfo/nanog
End of NANOG Digest, Vol 39, Issue 46 *************************************
On Apr 15, 2011, at 1:41 26PM, Marshall Eubanks wrote:
On Apr 15, 2011, at 12:44 PM, Mark Green wrote:
Suggestion; once on the 'night shift' stay put for at least three months... Sleep patterns take time to adjust. Jumping between day and night shifts will burn out even the most motivated employee.
What we found was that we would find people who wanted to be on the night shift, and would NOT like to be changed, at all. Some people like night work, or have family situations where it is ideal for them.
Yah. Read the current news coverage about sleeping U.S. air traffic controllers, especially the articles about how hard it is to switch shifts, and very especially if you do it often. --Steve Bellovin, https://www.cs.columbia.edu/~smb
What we found was that we would find people who wanted to be on the night shift, and would NOT like to be changed, at all. Some people
like
night work, or have family situations where it is ideal for them.
Regards Marshall
+1 I would start by first taking an audit of skills. Some people are just really good troubleshooters above and beyond average in that respect. You want at least one of those on each shift. Some other people are really great at attention to every little detail in documentation. You want at least one of those, too, on each shift. Sometimes those skills overlap but my experience is that they are rooted in different personality traits and the two complement each other and are only rarely found in the same person. Everyone else will be pretty much average in both respects. Then look at family situations. Married with children will probably not much care for swing shift if their kids are school age as they will never see them except on their days off. Mids are difficult for people with a toddler at home (ever try to sleep with a toddler in the house?) but work well with school-aged kids (parent can sleep while child is at school). Single parents are going to hate mids and swings. Look at individual preferences. Some people are natural night owls, some are natural morning people. Don't try to work against that if you can avoid it. So you might have a swing shift loaded up with single people, mids with people who like working those hours and maybe married with school-aged children. Day shift with single parents and higher level supervisory rolls. But the extent to which you take into account people's natural preferences, natural strengths and weaknesses, and their situation at home can make a huge difference in a harmonious situation on the job.
participants (10)
-
Chad Dailey -
George Bonser -
Jay Ashworth -
Jeff Shultz -
Mark Foster -
Mark Green -
Marshall Eubanks -
mikea -
Steven Bellovin -
Vinny Abello