Releasing code, time to stress out!

It is Friday and the deadline is today, your release is nearly “done” and we have to get this to production now. You are making preparations with the OPS team to get the release out. Because you did a vital change, the release is scheduled in the evening.

Then you go back to the code, oh, forgot got to fix this thing and forgot to merge these branches, push the code. Oh, somebody pushed something else and they touched a lot of my files, now I have to fix the merge conflicts.

The deadline comes closer…

In the meantime Ops is asking you for the release notes, but you are not done yet. You fixed the merge conflict and pushed your code. You ask Ops to move it to the test environment, you check it on test and it doesn’t work, you start hacking around to make sure it works on test. You tackled all cases, and thank god you are done just in time. Ops has its release notes, on how to release, they will release it at night. It’s 20:00, you can finally go home.

It is 01:00, Ops calls you, you wake up, they have a problem with the migration script, and it stalls releasing to production. Sleepy as you are, you open your computer and start looking at it. You figure out, it has nothing to do with you, and you call your colleague who was also releasing his code. Both off you try to figure it out, quickly fix it, so that the release can go through. You wait for ops to confirm the deployment went well, it did. All of a sudden there are http errors on production, you look at the logs and you see you are missing some configuration, quickly you adjust it, and production seems to work. Its 03:00 you can finally go to bed.

Does this stuff sound familiar?

You are doing it wrong! Stop wasting time!

Life is short, too short to actually be concerned about these issues. Instead of being drained of time and energy, you should be having a Friday evening dinner with your kids and feel energetic on Saturday. And the worst part of it, people are actually proud working like this, they feel important, they feel needed.

This process is error prone, repetitive and eventually leads to frustration, burnouts and depression. Be honest, nobody wants to stay up at night and do tedious error-prone processes, right?

Solve it!

The way to solve your problems is continuous deployment, there is nothing more satisfying than committing code on Friday at 17:30, pushing it to production and going home without a concern.

A lot of people are in the situation sketched by the example, they either don’t know how to solve the issue or do not have the means to address it. In the example there are multiple problems at play:

Ops has ownership over releasing. Development should be responsible, this forces you think about the release automation or removing deployment pain. Ops should focus on stability.
Feature branching, do trunk based development, this removes problems like “code locks” and reduces merge issues.
Big bang releases, work with small code batches only, make sure they are rigorously tested and deploy batches to production. This reduces deployment pain and stress. If something is broken, it must be from the last small change deployed.
Automate Deployment, deploying code is a repetitive task, computers are good at that.
Configuration Management in version control, When its in version control, you are able to run automated tests on it, to see if you system still works with the latest configuration change.
Build quality in! Write tests for everything, it is cheaper to detect issues with your code beforehand, fixing problems afterwards is expensive.

Most of the times there are way more issues then the ones elaborated in the example. Think about broken communication, politics, infrastructure.