Which DevOps CI/CD tools are the best fit for your development roadmap?


Continuous integration and continuous delivery are fundamental to DevOps. The aim is to speed up the production of code that’s in a fit state for release by standardising procedures, maintaining strict version control and minimising the number of serious bugs.

CI/CD tools are therefore one of the most important pieces in the DevOps tools jigsaw, and thus liable to influence other choices.

What are continuous integration and continuous delivery?

Continuous integration (CI) describes the collaborative process by which the work of a number of developers on a particular process is merged with the main codebase. The word ‘continuous‘ indicates that this is considered a flow rather than a series of steps; indeed integrations may happen many times a day or even many times an hour if you happen to be a Facebook or a Twitter. A CI tool seeks to automate many of the repetitive, time-consuming tasks in this process including testing the code before it is integrated and building the application.

These days CI is frequently conjoined with CD - continuous delivery. This refers to the practice of ensuring that there is always a repository of new code that is in a deploy-worthy state. Whether the team actually deploys is their choice, but the option should always be on the table. Continuous Deployment moves this process to the next stage, automatically deploying the code into the production environment every time a developer changes code, just so long as it passes all the automated tests. This is still fairly uncommon as for most organisations the risk of releasing buggy code outweighs the advantages.

A newer practice is progressive delivery. This is where new code is rolled out first to one group, then so long as all goes well, to other audiences, building learning into the rollout process and minimising the chance of an adverse reaction.

Other includes Nevercode, VMware, JFrog Shippable, Screwdriver, Semaphore, Buildbot, CodeFresh, AppVeyor, Spinnaker and Concourse.

Click image to enlarge

The top five tools

The granddaddy of CI/CD tools is the open-source automation server Jenkins, which has been around since before the word DevOps was a twinkle in Patrick Debois’ eye. Jenkins is free to use and open source, and still very widely used. With 1,000 third party plugins and counting it’s a true developers’ Swiss Army knife. But some of those we’ve spoken to say this can be an issue. Where do you find the right blade for your purposes and how can you know it’s any good? This means you’ll need some decent engineers in house.

In a Delta research study among more than 200 IT professionals, Jenkins was the name that came up most frequently – and the community version rather than the enterprise packages offered by CloudBees. It has a thriving and helpful community, and there’s now a separate cloud-native version, JenkinsX.

The second was an interesting one – GitLab. Perhaps best known as a source code repository and version control system rivalling GitHub, GitLab is, in fact, an end-to-end a DevOps platform designed to cover all stages of the development lifecycle while also facilitating collaboration between teams. Users liked its slick packaging and user interface, but some felt the pricing was a bit high for small projects (although it’s free for open-source codebases). GitLab plans to go public this year and told us there are many new features on the roadmap including feature flags, progressive delivery and evidence collection – logging the metadata around a build.

Speaking of GitHub, Microsoft has been no slouch in courting the powerful enterprise developer collective since purchasing the popular SCM. In November it launched GitHub Actions, a CI/CD tool that plugs straight into the GitHub repository. Already a sizeable number of our respondents said they had tried it and with GitHub claiming 37 million users there’s little doubt that GitHub Actions is going to be a serious player. Still early days, but given Microsoft’s clout, GitHub Actions could be a game-changer. Atlassian’s Bitbucket Pipelines is another CI/CD solution bolted onto a code repository.

GitHub Actions is actually a fork of Azure Pipelines, which is part of Azure DevOps Services (formerly Visual Studio Team Services, VSTS) a set of tools that run on the Azure public cloud platform. Azure Pipelines had been evaluated by 11 percent of our cohort, meeting with broadly favourable reviews particularly for its support for other cloud platforms including AWS and GCP and also Kubernetes. It will be interesting to see whether Microsoft continues to maintain the two forks or eventually merges them into one product line.

Just ahead of another cloud giant came Atlassian’s stalwart Bamboo. Unlike many of the tools here Bamboo is not a cloud offering (Atlassian provides that in Bitbucket Pipelines and discontinued the cloud version of Bamboo some time back) but it retains a loyal user base thanks to its ease of use. Some analysts have voiced suspicions Atlassian wishes to kill it off, but with a solid user base, it seems safe for now.

And then we have another cloud giant with AWS CodePipeline, Amazon’s fully managed continuous delivery service. For those already using AWS, this is an easy to use and inexpensive option that was especially appreciated for its security features in which achieved full marks. Those interested in a multi-cloud setup will want to look elsewhere though. Multi-cloud is not something that flies Mr Bezos’ kite.

The updated DevOps CI/CD Delta

The above is the tiniest flavour of what’s to be found in the fully updated DevOps CI/CD delta in which we consider the strength and weaknesses of the most prominent tools on the market, as well as some in the long tail too. Distilling the wisdom of more than 200 practitioners we aim to map the often confusing landscape of DevOps tools to make your choice easier. John Leonard, Research Director of Computing Delta will be presenting Delta’s research at this year’s Computing DevOps Live. This one day conference is free to attend for IT professionals – register today