Republished with permission from TheDailyWTF:
So we’ve just implemented a new change management process. As part of this, any changes to production systems need to be documented in full, and reviewed by a coworker from the same team and a manager before changes are made. All well and good, makes sense, no complaints. Well….maybe one.
I wanted to make a change to one of our mail appliances. My implementation plan was as detailed as it needed to be for any competent engineer – “back up the config. Change x from y to z. Commit the changes. Repeat on all appliances in the pool.”
I submitted it for review. It got rejected – not enough detail. I queried it.
Boss: It fails at the first step. I don’t know how to back up the config. I also don’t know where to find any of these values that you’re changing.
Me: I’m making the change, and I do know how to back up the config.
Boss: But what if you’re not here?
Me: I am here. I want to make the change right now.
Boss: Granted, but it should be step by step – What if someone needs to roll back your change?
Me: Everyone in the team knows how to back up the config, and where to find the values that are being changed.
Boss: But what if it’s not one of your team? You need to document exactly where to click or what to type, step by step, so anyone could do it.
Me: I don’t want anyone to do it. If it’s not one of my team, they’d better stay the f$#k away from my appliances.
– YES x 1000. I understand the need for recovery practices/DR/ISO, etc. but the key is getting someone who is familiar with the equipment/processes to be the one following the instructions and recognizing problems when the arise – i.e. knowing when to back off before totally hosing everything. – Rob
Picture Source: [Jeff Sandquist]