A couple years back, our workplace was converting some offices into clinic space. As such, a major demolition had to occur on the same floor as our server room. The good news: the construction crew did not have to do any work with the server room, nor interfere with any of the data cabling – the only thing they had to do was make sure our server room was putting out positive air pressure to keep all the dust out. Easy.
Apparently, one day the electricians were hooking into the circuit where our server room auxiliary air conditioning unit was also powered.
They neglected to turn the circuit back on…on a Friday night.
Now it almost goes without saying for the sake of this story that we didn’t have any kind of environmental monitoring set up in our server room at the time (after all, that’s why you are reading about it here).
We learned an important lesson very quickly.
When Monday morning rolled around, I came to work to a few people saying “The EMR is running slowly” – to which I thought “well, it always runs slowly.” However, as I rounded the corner to the server room, I heard the distinct sounds of many fans running at a speed I’ve never heard before. The doorknob to the room was extremely hot to the touch. Upon opening the door, an overpowering blast of super-heated air came at me – it took my breath away…the temperature? 116 degrees F.
Propping the door open with a bunch of box fans blowing the hot air out cooled things down within a couple hours – – No need to justify this purchase; we acquired an environmental monitoring appliance that week.
[Picture Source: eliduke (CC)]
I love the achievement unlock! I once walked into our datacenter within about 5 minutes of the air handler going down. Within 15 minutes it went from 65F to 97F. It was the weekend and I was the only one there. Being the new guy I was frantically calling my bosses asking what they wanted me to do. I ended up removing power from tons of equipment until we could get a heavier crew in there to get some sort of ventilation plan in place. Crazy how quickly data centers can get scorchingly hot.
Last year I walked into a pseudo server room and IT room where 2 full racks had disk arrays blazing away with 2 AC unit blowing directly into the back of them. The IT staff were the only ones allowed to wear shorts in the office 365 days a year because the average temp in the room was 90F. I asked why they didn’t at least move the 4 workstations OUT of the room and put more AC in and the answer was they didn’t have anywhere to move those 4 stations too.
I guess workman’s comp claims were cheaper to handle than renting more space for IT….
I worked as solo IT for an environmental lab which had separate airconditioning for their GC (Gas-Chromatograph) room. There were 6 GCs in there, all running overnight with internal “Ovens” which reached 300C (572F). There was a power outage (the GCs and computers were all on a UPS with backup generator) and the aircon didn’t come back on automatically like it was supposed to needless to say, the next morning we came in to find the GC room at 60C (140F) and a hell of a lot of screaming from the computers. Luckily the only damage done was a hard drive died in a computer running win 2000 that was on its last legs anyway. 😛