Screwing up badly at work

I changed a configuration on a client's site that triggered thousands of emails to different customers, which forced our client's Customer Service team to handle the same number of customer calls in a single day that they would usually receive over a normal week. They even had to write an apology email for all affected users, assuring them there was no data breach on the site.

It was a long day.

Admit it

Shit happens, and people make mistakes. I did on that occasion, and I will again in the future, or somebody else will, but at least I'll be happy if we, as a team, don't make the exact same mess I did again.

The first thing to do when you screw up is to admit it: tell somebody that you screwed up and explain exactly how.

If there's a big problem, and you alone caused it, chances are people will be chasing ghosts to fix something they won't fully understand if you don't come forward and tell them exactly what happened.

You are the only one that knows, basically (until everybody finds out).

The goal is to shift everybody's mindset from panicking about a mystery error to focusing on putting out the fire and doing some damage control. Back to my example: everybody was wondering how the emails went out, but when I told them it was me who had changed a setting, everybody moved towards finding a solution (emails were still going out at that point).

Trying to cover up something is really a stupid idea because, again, shit happens and it shouldn't be the end of the world. In any case, people will find out rather sooner than later.

Say sorry as you should be

Being on the defense on this type of situations is pretty common, but you shouldn't be. You made a mistake, you admitted it, now apologise without making excuses or blaming something (or somebody) else.

As individuals and as a team we should learn some stuff starting with the fact that human error is an actual thing. Saying sorry is as important as learning to accept someone's apology.

Keep in mind that if you didn't screw up this time, you could be the one causing the mess next time, so don't believe you're so perfect. At the same time, if you were the cause of all the problems today, take a breath, somebody else will take that leading role next time.

Not the same mistake twice

When the adrenaline is over and the problem is solved do everything within your reach to avoid the same mess to repeat itself.

Let's try to have new problems, not always the same ones (it's boring otherwise).

While all human errors can't be eliminated (unless there's no human in the equation) we can always reduce the chances for them to happen by identifying what mechanism we (the team) can put in place to prevent them.

Back to my emails: the setting I changed wasn't on the Live site but on a testing environment that happened to contain real customer data. In this case, the problem was in how we create those testing environments (a fault in the process we have in place).

The problems could be prevented by improving the processes, adding any necessary documentation, and most important by spreading the information across the team.

Learning from a mistake isn't a cliché; it's something tangible.

If you really learned from a mistake, you will end up with more documentation available to the team, a better process in place, and everybody informed about what happened and what can be done to avoid it happening again.

Always keep in mind that, in 2017, La La Land was announced as the winner of the Best Picture category at the Oscars because somebody handed the wrong envelope. And they improved the process afterwards.