DevOps Best Practices and Considerations

Most likely, you’ve heard something about DevOps. Over the last several years, the topic has gained momentum, as many organizations attempt to improve their software delivery speed and reduce some of the manual, error-prone tasks that are involved when getting software to production.

But, most of the problems these organizations are trying to solve have been in the software space for decades. In ancient history, software much simpler than what we are developing now took a long time to code onto punched cards. Since then, the software engineering industry has evolved dramatically to make the act of writing software faster, and simpler. We now have compilers, IDEs, frameworks, emulators, and many other tools that have accelerated and facilitated aspiring engineers to get into the world of zeros and ones.

So, why is this still not fast enough? One of the main reasons I see is as soon as we gained traction with creating and delivering software, we entered into a global competition of expectations. We want to be as fast as the unicorns and tech giants that deploy to production thousands of times a day.

Delivering software quickly and correctly to users has always been a desire because it can lead to excellent competitive advantage. This demand isn’t new; even ten years ago, I worked with clients that wanted to get features faster than the usual three- to six-month release timeframe to keep customers happy.

This push to be the first or the fastest is the reason the promises of DevOps have caught the attention of business leaders. Today, companies in literally any part of the world can pop up and forge completely new industries, or disrupt and take over your business.

But DevOps is not magic. Increased associated costs (either opportunity or financial) can make your organization have a hard time if a DevOps initiative is not planned correctly.

1. DevOps Requires a Cultural Change

One of the main things that I’ve noticed when organizations are interested in DevOps is that they forget about culture. Sometimes the top IT head is interested in implementing a strategy, but ends up trying to force change upon a team that is “happy with how we do things here.” Changes are painful, and for a DevOps strategy to work, organizations need to make many adjustments both in practices and structure. Below are some I’ve seen most challenging to complete.

Adjust Team Structure

One of the main benefits of DevOps is removing barriers between teams. This means that instead of having the traditional silos (IT, Development, QA, Business) teams are organized in a way that works best for the organizational goals, and not just for each department.

There are multiple topologies for doing this, but in the end, all focus on avoiding the traditional throwing-work-over-the-wall culture where each team (silo) is responsible for only a part and letting other groups downstream handle the problems.

Re-Define “Completed”

This is a common issue in many organizations. Development passes something “completed” to QA or IT and then forgets about it, sending the message that “it’s not our problem now (unless you file and prove the bug properly).” This situation is especially problematic when working with teams with a big timezone difference.

Completed, on DevOps terms, means working in production. Organizations need to track work differently than what teams are currently used to. It’s of no value having a bunch of user stories “completed” by the development team when they get stuck in the queue because there are no environments to test them properly, or the next release window will be in 7 weeks.

Eliminate Superheroes

Sometimes organizations depend on a single person to solve critical problems. Maybe that person is a developer that has more time working in the application or someone that happens to know more about the environment. They might be able to solve the urgent problem but become a bottleneck that will turn into a significant risk for any organization.

Knowledge needs to be shared and kept up to date in a way that is pragmatic, useful, and efficient.

Reduce batch size

As developers, we tend to stay working on something for a long time. Either because we want it to be perfect, or just because we are “in the zone” and simply don’t want to stop. But, if you aim for deploying software to production multiple times a day or week, we need to be able to reduce the size of the pieces of code we create. Additionally, having smaller batches helps reduce the risk of conflicts when merging with the work of other developers. The time to test gets also reduced, and the risk of causing a global failure when deploying to production gets diminished as well.

Quality Should Already be a Priority

Many companies complain about software quality but don’t do much about it. QA teams are not part of their core software strategy, and sometimes even practices like unit testing are not in place.

When this is the case, a DevOps strategy most likely won’t get traction, because the promise of having fast feedback for the team (a build failing, a test not passing) is not fulfilled. Testing only at the end of each sprint is maybe late.

If you need to have a full regression test on production, possibly it is because you don’t have enough confidence that your development life cycle helps you to deliver quality software in each release.

2. DevOps is Not (Just) Automation

Another DevOps best practice is understanding that while automation is a good part of a DevOps strategy, it is not the main one. It is more a means to an end than a goal in itself. If proper cultural changes are not in place, automation will not make much difference. Yes, you may be doing some things faster, but still, the constraints will be somewhere else.

As Dr. Eliyahu Goldratt describes in his Process of On-Going Improvement, you need first to take over the constraint and look for ways on how to get the maximum benefit of it. If you have waste in any of the steps (like developers waiting for a DBA to perform/authorize a change, or QA team not being able to move on because development is late in their work), automation will be just another waste.

3. You Need to Embrace Chaos

One of the philosophies championed by the Lean movement is Fail Fast. Or, paraphrasing Mr. Zuckerberg, we need to “move fast and break things.”

Part of being able to get changes to production fast is that things will go wrong sometimes. Notice, I didn’t say “things might go wrong.” Culture and automation changes described above (among other DevOps practices) help to reduce the risk of catastrophic failure, but there will be failures anyhow.

The point here is that we need to be able to learn (quickly) from each problem we have, and once we’ve made all the changes required to move fast, you can solve most problems at the same pace.

But many organizations are not yet ready to embrace chaos and failures. Riks are often transferred to vendors, or changes are reduced to a minimum so we can avoid it.

Not All Organizations Are Ready for a DevOps Strategy

It’s not that a DevOps transformation can’t succeed in cases like mentioned above. There are multiple examples throughout the last few years that demonstrate that even organizations that are enterprise-rigid can benefit when everyone is involved and include the required culture and organizational changes in the strategy.

Using the latest tools or the shiniest development practices won’t make you get a successful DevOps transformation. There are some investments you need to consider before embarking, so it’s better to think about them before launching the ship into the water.

[adinserter name=”DevOps-Closing”]