Next up: The fire department gets a call about the firewall…

I remember about 6 or 7 years ago I worked for a computer repair centre. We used to have a ‘wall of shame’ in the tech room, where basically if any of us came across a particulary brain dead client we would stick a copy of the work order on the wall for the rest of the employees to see.

(A bit of background, the client was a big corporate client of ours but they were working out in the middle of nowhere. It was a 6 hour round trip for our technicians at $90 per hour travel time, and a minumum charge of 3 hours on-site time at $150 per hour at a priority service charge)

My favourite one on the wall of shame went something like….

WORK DESCRIPTION

Client called to report that one of his servers is making a beeping noise. This call is urgent. He authorizes a priority service call for repair.

RESOLUTION

Advised client to replace batteries in fire alarm

CHARGES

$990

[Picture Source: shingst (CC)]

Computer support can only do so much…

This was during the early 2000s, when things like spyware and malware was really still in the ‘OMG’ phase. I get a call from a really desperate sounding man.

Me: Thank you for calling (broadband company) My name is Daniel, may I have the phone number on the account, area code first please?

DC (desperate customer): Oh thank God! Ya gotta help me!!
Me: Yes sir, just give me your phone number, and I’ll open your account
DC: She’s due home in an hour!!!! I’m in so much trouble!!!

After much wrangling and finally getting his phone number, I ascertain the problem: This gentleman had received high speed internet two days before his wife left for a business trip for two weeks.

Evidently, he’d done absolutely nothing the entire time she was gone except surf porn. Lots and lots of porn. Lots and lots of porn = Lots and lots of spyware, malware, adware, viruses, etc…to the point where his browser had been hijacked, his computer would barely run, and he was getting pop ups when his Internet explorer wasn’t even running. Usually in these cases, I am required to use my best judgement. i.e. – foist him off on someone else as *their* problem. But I was wanting to test out my mad skills, so I set to work.

Anti-virus program? he didn’t have one, install one! Anti-spyware? didn’t have one. Let’s install one! Let’s install two! he found 29 viruses, and 351 pieces of spyware.

Feverishly, we worked together, restarting the computer over and over, slowly chipping away at this veritable porn paradise. Finally, with fifteen minutes to spare as to when his wife was due home, we’d scrubbed his registry clean and got his computer all squeaky shiny again.

Feeling proud of myself, after he thanked me profusely, I hung up the call and thought nothing more of it. However, friend a of mine at work stuck his head over the cubicle a couple months later and said, “Uh, hey, did you work with some idiot who kept looking at porn on his computer?”

“Uh, yeah, why?”

“Dude’s on the phone. Guess he jacked everything up again, and his wife wants a divorce.”

*facepalm*

[Picture Source: jamaisdeux (CC)]

Contest Upate

The voting stage of the made-up IT term/phrase is here! Cast your votes now becasue it closes on Friday at Midnight!

Prize Update: We had to change the prize. Our legal team (imagine two bald IT guys and one with hair) have come to the conclusion that we cannot give out food items as prizes. Instead the winner will receive an awesome Fail Desk Tee-Shirt.

[polldaddy poll="6130295"]

IT is also about helping…we don’t always hate our users

This comes from one of our female readers, so the last story here has a bit of a different feel… – Rob

For the first few years of my professional IT life, I was the jack-of-all-trades for a small business, including all sysadmin duties on top of developing and maintaining all aspects of the website, and being in charge all marketing for it as well.  I decided to switch jobs because I wanted to have a life, and jumped to a large web development shop to narrow my focus.

As I was (and still am) more interested in user experience and front-end development, I ended up in the “Creative” part of the company that developed the front-end.  So it was populated with UX’ers, HTML developers and graphic designers. When word got out in the department that I actually understood the mysteries of computers, my new colleagues would quietly check with me first before embarrassing themselves with our internal support desk.

In the ensuing months, I patiently showed them how to remove the lint from their mouse rollers (this was the early 00’s), to check for the Caps Lock light when they couldn’t log in, to check their cables (the cleaning staff regularly loosened cables with less than gentle cleaning) and to make a circuit of our three network printers to make sure they had selected the correct one and/or there was enough paper and toner before they declared the printer wasn’t working.  The IT department learned to love me since their support calls dropped off dramatically (and also because I regularly brought them offerings of triple-chocolate espresso cookies ;) ).

But my most memorable experience was when I came in early one morning, and found one of the graphic designers – a really nice but scary looking pierced/tattooed monster of a man who looked like he belonged in a motorcycle gang – sitting at my desk near tears.  Knowing I came in earlier than the other Creative staff, he had been waiting for me for the past 45 minutes  He had come in 2 hours earlier to finish up a project that was due by 12 noon, and “couldn’t get his computer to work”.  The screen was “all black” no matter how many times he rebooted.  The computer was also making “funny chugging sounds”.

As we walked to his desk, I started asking the usual questions about cables and power cords and such – he had checked everything.  I was beginning to think that maybe the hard drive was actually fried until I sat down and powered up his computer, saw the POST message and then the “chugging” started.

I shut the computer down and ejected a floppy disk from the A: drive.  The poor designer’s face was vacillating between utter relief and absolute mortification. I quickly rearranged the boot order and assured him I wouldn’t tell anyone. (And haven’t until today, over a decade later ;) )

[Picture Source: leyla.a (CC)]

Infected Mouse Make Bad Trades

I was a stockbroker at a popular online brokerage. In order to save money, the brokers in the branch offices were made into the frontline tech support for customers. Needless to say, we weren’t trained beyond a “really helpful” wiki on our corporate intranet. 99% of the calls were easy “reinstall java” or “clear your cache” type calls with the 1% falling into ugly thing that involved checking IP addresses, log ins, etc.

One of the 1% calls I got was from an elderly man. He called up and asked to speak to tech support. He needed a series of daytrades that day “busted” which means we as a company would eat his losses. He proceeds to tell me that they were placed due to a virus that was in his mouse. He actually argued with me that the virus in his mouse was filling out the web order forms and doing the buying and selling. Mind you, it only was doing the bad ones, none of the good ones, he did those. I escalated the call to the help desk and the next thing I know the man is in my office holding his mouse telling me that tech support requested that he bring his mouse in for inspection.

It wasn’t even a USB mouse. It was an ancient serial mouse. We “examined” it for a couple days and stated that we could find no evidence of the virus. We were nice enough to include a free copy of anti-virus software as a courtesy. Guess who got to go to the customers house to install it?

 Infected mouse, never hear that one before. -Scott

May cool heads prevail. Now make me some coffee.

As is usually the way with these things, a year-end upgrade was planned out for our full-disk encryption software to upgrade it to the latest version.  In a perfect world, it would uninstall the old version and install the new version.  It was tested on several machines in our headquarters only, with no adverse effects.  A gradual rollout was scheduled to be done AFTER the encryption administrator came back from a short vacation.

The day after the administrator left on vacation, we started to get laptops coming in from across North America with the new version of encryption on it.  The machines themselves were being shipped to headquarters for reimages due to various issues, but not the encryption itself.  We groaned to each other, because this is par for the course for us, and made ourselves a note to say unfriendly words to our administrator when he came back.

Our administrator returned from vacation, and admitted that SOMEHOW the encryption upgrade had been pushed to 3500 machines outside of headquarters.  He assured us that he would halt the rollout, and that machines that hadn’t begun to install the new version would not be upgraded.

Several days later, we started to see machines come in with new version of encryption installed, but with a twist.  The encryption software would not boot.  We took one over to our administrator, who tinkered with it, concluded that he couldn’t do anything with it, and that we should reimage it.  This lasted for several hours, until we noticed that the tickets indicating a machine inbound for reimage had ballooned, all reporting the same fatal encryption error.  We quickly realized that we would be receiving several hundred of these machines in the space of a few days, when our normal volume for reimages is only 10-20 machines per day.

Cue the panic.

We bluntly informed the powers-that-be that we had neither the physical space to stage several hundred machines, the manpower to reimage several hundred machines (at least, not quickly), or the server space to perform data backups (as we usually do whenever we reimage).

The powers-that-be quickly responded, getting us several conference rooms set up for reimaging, and getting more than a dozen IT employees to volunteer for some crash-course training in reimaging.

However, the server space turned out to be a problem.  We had at our disposal a PowerEdge with a 1.3 TB RAID array and a normal desktop computer with a 1 TB hard drive.  Normally, we could make this last for about two weeks before we had to start deleting old data to make room for new data.  However, our Security group firmly insisted that we needed to retain all the data from machines afflicted with this encryption error.

The other IT employees immediately began plumbing the depths of whatever resources they had.  Someone offered to archive old data on their group’s network share, another person donated a legacy server and configured it with a 900 GB RAID array.  However, we knew (and told the powers-that-be) that we needed way more than that.

Amidst everyone else running around like beheaded poultry, our chief security officer (who happens to be one of the developers of BackTrack) walked up to me and asked about how much space was actually needed.  I ballparked high and told him 20 TB.  He nodded, grabbed his coat, and walked out.

Thirty minutes later, he returned from Best Buy laden down with bags full of 1 TB external hard drives.  He had driven to the nearest Best Buy, walked up to the customer service counter, and casually purchased their entire stock of them.

And he bought a coffee maker.

Skill level: executive

I work as a Sys Admin at an Ivy League college for my department only.  Our department manager thinks we do nothing but carry around keyboards and usb keys; she has stated as much to my supervisor.  She is a mean, cold, evil woman who can’t remember her own name half the time.  The other day I get a phone call (instead of an email address, because she can’t remember to email support instead of us individually) that she can no longer “hear sounds.”

She stated that she went into the control panel and her sound settings appeared to be ok.  I have no idea how she remembered to do that, but gave her credit and told her I would come take a look.

As I walk in her office, I can see that her speakers are off (no light).  So I ask her to please turn on her speakers (as she sits in her chair and “guards” her computer unless you ask to touch it).  She looks at me with her usual deer-in-headlight look and says “how?”

I ask for permission to reach around her and quickly turn them on and leave…BEFORE I fall over laughing.  As it happened, her friend and partner-in-crime (the HR director) who sits two doors down from her had the EXACT same issue a week before.

[Picture Source:Brian Lane Winfield Moore (CC)]