Ricky Beam wrote:

On Fri, 01 May 2009 21:32:19 -0400, William Warren <hescominsoon@emmanuelcomputerconsulting.com> wrote:

...

...

Specifically, I am using the guide posted at: http://www.openxtra.co.uk/articles/calculating-heat-load

"Before you decide on an air conditioning unit you should commission an audit from a suitably qualified air conditioning equipment specialist or installer."

Translation: Hire a f***ing professional.

And that's exactly what you need to do. Qualified HVAC installers (with specific data center experience) will know far more than us "network types" will ever want to know about cooling. They do this for a living, and thus, know all the tiny details and odd edge cases to look for. (like looking above the drop ceiling -- that's what it's called, btw -- and seeing what's up there long before pencil meets paper (not that anyone uses paper anymore.))

...

You also have to take into account the environment surrounding the data room. At my wife's work The ceiling above is only separated with a false ceiling to the metal roof above but the rest of hte spaces surrounding the room are climate controled. They [had] to significantly upsize to account for the heat load of that ceiling.

Unless you are pulling air through the plenum (that space above the drop ceiling), the air up there shouldn't matter much -- there should be plenum returns up there to begin with venting the air to the surrounding plenum(s) (i.e. the rest of the office, hallway, neighboring office, etc.) However, I've seen more than enough office setups where the "engineers" planning the space completely ignore the plenum. In my current office building the static pressure pushes the bathroom doors open by almost 2". And they placed our server room directly under the building air handlers -- meaning all the air on the 3rd floor eventually passes through the plenum above my servers. (also, the sprinkler system riser room is in there.)

The space above the drop ceiling is only a plenum if it's used as air handling space opposed to ducting the returns everywhere. If it's not an air handling space, it's not a plenum, it's just where spiders might be. It's easier to throw grated panels in all over the place for returns in large systems. Now, back on topic, plus nifty graphics explaining the difference: http://en.wikipedia.org/wiki/Plenum_cable

Bottom line, again, ask a professional. NANOG is a bunch of network geeks (in theory.) I'd be surprised if there's even one licensed HVAC "geek" on the list. ('tho I'm sure many may *know* an HVAC engineer.)

But yes, please, don't learn how to make your own system from what we say here. HVAC systems are their own world. You wouldn't want an HVAC guy designing your network just because he's seen a lot of server rooms, would you? ~Seth

On Sat, 02 May 2009 00:26:37 -0400, Wayne E. Bouchard <web@typo.org> wrote:

... approximation

Even an approximation is hard to make. One might think the simple math of "how much power is fed into the room" would do, but it ignores numerous factors that greatly effect the answer. I can rattle off example after example, but it's unnecessary. You'll need professionals to install the hardware, so there's no point not calling them in for a consult. (And the elephant in the room no one has mentioned is "air flow". Cooling capacity is only half the equation. Air flow *volume* is just as important.) --Ricky

On Sat, May 2, 2009 at 12:06 AM, Ricky Beam <jfbeam@gmail.com> wrote:

Translation: Hire a f***ing professional.

And that's exactly what you need to do.  Qualified HVAC installers (with specific data center experience) will know far more than us "network types" will ever want to know about cooling.  They do this for a living, and thus, know all the tiny details and odd edge cases to look for. (like looking above the drop ceiling -- that's what it's called, btw -- and seeing what's up there long before pencil meets paper (not that anyone uses paper anymore.))

Ricky, You shouldn't even *have* a drop ceiling in a modern computer room. You want the room to be as tall as practical so that the air from the hot aisles has somewhere to go on its way back to the HVAC, other than back through and around the cabinets. Personally, I've found that there's a pretty wide disparity between HVAC professionals that are capable of hooking up a CRAC and turning it on versus HVAC professionals that understand the holistic picture including hot aisles, cold aisles, humidity control and flow. I wouldn't want to call in a professional without first understanding the problem well enough to assess whether I was getting a competent answer. Regards, Bill Herrin -- William D. Herrin ................ herrin@dirtside.com bill@herrin.us 3005 Crane Dr. ...................... Web: <http://bill.herrin.us/> Falls Church, VA 22042-3004

Below are a few snippets from recent posts, in no particular order, that had me saying to myself "does not anyone remember an interesting alternative I thought had come up on NANOG a few years ago?" Well maybe it was some other list, but it is not really worth going back and looking. It isn't quite true, or totally wise, but you can EASILY ignore HOT and COLD isle systems, and you could even have adjacent cabinets in any row blowing opposite directions and randoml;y facing across the isle to a cabinet in the next row that could be facing EITHER WAY, and you need not care! (don't really do it, BUT YOU SAFELY COULD) No raised floors needed, unless you need to for cables, and relatively low ceilings and ladder racks / cable trays with massive wads of cables that would block normally required air flow being no problem at all. And NO it isn't chilled water to leak and destroy your equipment Oh, and at maybe only 30KW capacity per cabinet (with an extra very REAL 50% reserve capacity) - is that enough for you...?? And no problem with random cabinets or multiple whole rows with no equipment yet or even just turned off until they pay their bill. Nothing freezes, and nothing roasts. Just works. All the pieces I snipped below are about trying to get the heat from the cabinets to the CRAC and the cold air back with as little mixing and possible. The more you mix, the more total air you have to circulate and the lower your efficiency goes. *** start of snips *** (And the elephant in the room no one has mentioned is "air flow". Cooling capacity is only half the equation. Air flow *volume* is just as important.) You shouldn't even *have* a drop ceiling in a modern computer room. You want the room to be as tall as practical so that the air from the hot aisles has somewhere to go on its way back to the HVAC, other than back through and around the cabinets.

I love my 30' ceiling. Even with all the things that are wrong with our HVAC setup, the servers survive due to that ceiling.

...

... versus HVAC professionals that understand the holistic picture including hot aisles, cold aisles, humidity control and flow. I wouldn't want to call in a professional without first understanding the problem well enough to assess whether I was getting a competent answer.

The return feed is below the units pulling ambient air, and the cold air is injected 15+ feet above the isle behind the servers, intermixing with the hot air as it rises up the wall.

At least it works, but it could be better and changes will need to be made before I can reach 50% capacity in the racks.

*** end of snips *** A few simplifying ground rules. An existing "dumb-ass" grade CRAC system can be used to control humidity and to do any required fresh air changes, etc. With cabinets of electronics, we are ONLY talking about a sensible load with the stuff below. We DO NOT WANT or need to deal with humidity with the system I'm referring to. It is designed to JUST remove sensible heat VERY VERY efficiently. Its refrigerant lines, both supply and return, are almost at room temperature. No sweating, no dripping, no insulation really needed! Piping from the mechanical room is designed for low pressure drop for efficiency, but this refrigerant, R744, does a lot of cooling with very small pipes. The regular refrigeration systems we are used to have oil circulating with the refrigerant and special care is needed to ensure its return to the compressor. Also, the traditional system needs a thermostatic expansion valve (TXV) so the evaporator has liquid refrigerant until almost the end of the coil, but no liquid is returned to damage the compressor. A typical TXV may have a MOPD (minimum operating pressure differential) of 100PSI. In some cool outside weather conditions, you artificially throttle fans or even bypass some of the condenser coils to keep your head pressure up to keep that 100PSI MOPD so you get adequate liquid flow through the TXV and don't starve the evaporator (which cuts its capacity and can lead to icing that can progress across the face of some coil designs and then blocks air flow). But keeping that 100 PSI in mild weather is inefficient, too. Anyway, with NO oil circulation, and NO compressor in the loop at all and NO TXV needed, you have something that is very close to a two pipe steam heating system with a condensate return pump for the boiler. Typical of many buildings and even some larger homes especially some years ago. Here, however, the "BOILER" is a finned coil on the back of each of your cabinets. This coil is fed from a local manifold via ball valves, excess flow safety shutoff valves, and flexible metal hoses. The finned coil equipped rear door also has its own fans. It gets raw undiluted hot air exiting your equipment, passes it over coils loaded with a liquid refrigerant just below room temperature and AT a system pressure that any additional heat added will just boil off some of the liquid which will be entrained as bubbles in the slurry. The cabinet air exiting the coil-door's fans is AT room temperature. You have enough excess liquid R744 flowing that you could handle a 50% overload beyond rated capacity. This excess liquid flow costs almost nothing as the piping is all designed for low pressure drop and it is a low head pressure centrifugal pump that keeps each evaporator swept with enough liquid for 50% overload and yet there are NO ISSUES at all with partial or no load cabinets. No super cold spots, etc. Instead of the "radiator" in your house, the condenser here looks like a classic shell and tube heat exchanger where the returning refrigerant gas and liquid slurry is simply dumped into the shell and any entrained liquid drops to the sump at the bottom or even to a seperate "receiver" tank below if so equipped, and the returning gas simply is condensed on the outside surfaces of the tubes in the tube bundle. These tubes themselves can be cooled by chilled water that may exist in many big buildings, or may be DX coils for any traditional refrigeration system that comfortably can be modulated to run over a wide range of loads and always keep the R744 just below room temperature. There is even a simpler system these folks make that doesn't even have the R744 pump, but depends on the "shell and tube" condenser and receiver to be physicall enough ABOVE the cabinets that gravity adequately feeds the cabinets (those with no heat load will simply have their coils stay full of liquid) and only gas goes back to the condenser. This is a great way to have you building chilled water system do the work for you and yet you can keep water piping offset to the next room and so OUT of your datacenter (could be on the floor above if that next room isn't high enough). This is still the classic 2 pipe steam system but all gravity return with no condensate pump needed. So why isn't everyone doing this? TOO DARN EXPENSIVE! - needs competition. Europeans selling un USA. "NOT INVENTED HERE"? And, of course, R744 is simply CO2! At a little over 80F liquid CO2 gets to 1000PSI. The piping for this needs to be done by folks a bit better trained than the average refrigeration guy. Some would be fine, but many need a little training. Thick walled copper could be used, but I think all this datacenter CO2 stuff is being done in welded stainless steel and any unions would be bolted flanged connections. Properly trained datacenter staff can valve off and simply VENT (slowly) off any remaining CO2 and remove a leaking or whatever door assembly that needs to be changed. Normal chassis fans and if needed a floor fan in front of the cabinet will move enough heat out of the cabinet so it keeps running and the inlet air for a few adjacent cabinets is now a few degrees hotter but their exit air is ALL at normal room temperature. The rest of the room isn't impacted at all. The replacement door gets connected, refrigerant valve cracked at one end lets in R744 to sweep out any air to a vent you open at the other end of the coil (unlike regular refrigerants, this is LEGAL!!!) vent plenty excess, no problem (you MUST NOT ADD AIR the the big system). Seal it up, open bothsupply and return valves fully, and you are back in business. Your equipment in that rack could have had its "cooling door" non functional for weeks if necessary with NO PROBLEM (assuming the others near by were all working). Real world high reliability configuration would have several systems running in parallel. Perhaps no two adjacent rows on the same system. Simple valving could allow sections of one normally isolated system to be shared with another or a built in spare in an emergency. There are none of the oil return to many parallel compressors issues. You simply DO need to be sure each running section has enough liquid CO2 to work and that could be easily taken from an overfilled other system or the LARGE on site refrigerated bulk thermos like storage tank typically running just under 300PSI you probably should have available, anyway. It might also do double duty as a CO2 fire suppression flooding system, but CO2 flooding can kill people if done wrong. Maybe the onsite soft drink system in the cafeteria already has a bulk delivered CO2 system that could be tapped in an emergency for some CO2. Clickable brochures and such on the lower half of this following page will be of interest: http://www.trox.co.uk/aitcs/products/CO2OLrac/index.html They are in the USA, too.

On Fri, 01 May 2009 21:32:19 -0400, William Warren <hescominsoon@emmanuelcomputerconsulting.com> wrote:

...

...

Specifically, I am using the guide posted at: http://www.openxtra.co.uk/articles/calculating-heat-load

"Before you decide on an air conditioning unit you should commission an audit from a suitably qualified air conditioning equipment specialist or installer."

Translation: Hire a f***ing professional.

And that's exactly what you need to do. Qualified HVAC installers (with specific data center experience) will know far more than us "network types" will ever want to know about cooling. They do this for a living, and thus, know all the tiny details and odd edge cases to look for. (like looking above the drop ceiling -- that's what it's called, btw -- and seeing what's up there long before pencil meets paper (not that anyone uses paper anymore.))

[...]

Bottom line, again, ask a professional. NANOG is a bunch of network geeks (in theory.) I'd be surprised if there's even one licensed HVAC "geek" on the list. ('tho I'm sure many may *know* an HVAC engineer.)

The problem is that you're ultimately responsible for determining the competence of that "professional." I've seen a number of them who have talked about their "data center" experience, when it turns out that their idea of "data center" is "I went down to $FooCo and set up their 200 sq ft of server room with a mini-split system". So, here's better advice. Take some time. Educate yourself. There's a ton of useful information out there that should give you some basic grasp of what's important and what's not. Airflow and BTU are important. Do not be afraid to get into it a bit: learn what "latent" and "sensible" mean in the HVAC world. Knowing about realistic layouts for data centers is important. If you run into a "professional" that looks at you funny when you say "cold aisle," you know that you can wind up the conversation and thank them for their time, and then call someone else. But when you find the right guy, you should be able to hold an intelligent conversation with him, he should be able to make plausible and reasonable explanations of where you've misunderstood things, and the best guys will not mind that you are trying to work with them closer to their level, because ultimately it means that their task is also more likely to be successfully completed, and that you'll be coming back to them in the future. Best to not just blindly hand things off to a "professional." Be sure that they do, in fact, have the skill sets and experience you need. Know too that they are relying on you for critical bits, and that the best situation for both parties is when you each understand the other's stuff well enough that you can actually get the right thing done. ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net "We call it the 'one bite at the apple' rule. Give me one chance [and] then I won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN) With 24 million small businesses in the US alone, that's way too many apples.