Every modern development team, along with using one of the Agile frameworks, at some point in the development process requires implementing Continuous Integration/Continuous Delivery pipeline. There is a vast diversity of solutions available today which help development teams to implement a working solution, but in this article, the author has decided to analyze Jenkins and CircleCi since those are the tools with which he has the most experience.

Both Jenkins and CircleCi are powerful automation tools, but there are significant differences between those two that might or might not suit the team. Jenkins as a solution is much more mature than CircleCi since it is available on the market since 2004, initially named Hudson, later renamed to Jenkins due to disputes with Oracle, while CircleCi is on the market since 2011, but only recently it is raising to its full potential.

But to understand why one solution is better than the other for certain aspects, let us dive deeper into the problematic.

 

1. Efficiency

Although Jenkins comes with a Docker support, it is much more complex to integrate Docker with Jenkins than it is to do the same with CircleCi, given that Docker is fully integrated into CircleCi and using specific Docker image is just a single line inside CircleCi config file.

Most of the Jenkins configurations will probably end as a solution where one master server communicates with various slave servers based on top of some common operating systems such as Windows Server, Ubuntu Linux, etc.

Although it is a much bigger effort to configure and maintain Jenkins pipeline, we will see excellent results in terms of the efficiency of repetitive tasks once we reach the desired point. This efficiency is because Jenkins remembers the previous state and has out-of-the-box caching since memory is not wiped in between jobs, which is the scenario with a Docker-based solution of CircleCi.

On top of this, CircleCi has a rather strange mechanism for caching docker containers between different jobs, which makes its execution time pretty unpredictable.

Winner: Jenkins

2. Reliability

Jenkins is a stateful solution which helps a lot with efficiency as described above, and this statefulness might also have negative effects.

It makes Jenkins pipelines prone to sporadic failings e.g., when the machine state wasn’t properly restored to the working point between jobs, one of the slave machines is down for maintenance or any other reason, License of some tools has expired, etc.

On the other side, because CircleCi is a cloud-docker-based solution without dedicated resources, it is also very reliable to provide the same results in between different jobs given that Docker image is reliable on its own.

Winner: CircleCi

3. Pricing

Jenkins is a fully open-source solution, but that does not necessarily mean that it is free. The fact that it (in most cases) requires a dedicated machine for hosting Jenkins server. Along with that, for more complex solutions, we will probably need multiple platforms with potentially different operating systems, which adds cost to the “hidden” pricing list of an “open- source” solution.

On the other side, even though CircleCi comes with a pretty good free plan, for any advanced usage, such as running multiple jobs in parallel, we will have to pay for some of the performance plans

Winner: Tie

4. Version control

Full version control of Jenkins pipeline is ranging from very hard to impossible. Jenkins is relying on a GUI based job-configurator in combination with a script written in one of the supported scripting languages such as Groovy, which is either embedded inside job configuration itself or stored in a git repository.

On the other side, CircleCi is using XML based configuration, which is embedded inside the same repository as the source code of the product itself. Every step of the pipeline is described inside the dedicated config.yml file.
The pipeline is being run automatically on each code push to git repository, and integration with a repository hosting service such as GitHub and Bitbucket is done with a single click.

The fact that pipeline code is fully embedded into the same repository with the rest of the code, minimizes the risk of misalignment of those two and significantly simplifies maintenance.

Winner: CircleCi

5. Integrations

One of the most significant powers of Jenkins comes from a large variety of plugins available. There are over 1500 different free plugins for Jenkins, which significantly help to make the pipeline better. There are plugins for almost any need, which substantially reduces an effort when implementing some features. Some examples of useful plugins are Mailer for sending e-mail notifications, Git for easy integration with git, Slack Notifications for integration with Slack, etc.

On the other side, CircleCi has many pre-built Docker images made in-house for integration with most popular programming languages, but it doesn’t have any plugin-based solutions that extend original functionally as Jenkins does.

Winner: Jenkins

6. Build configuration

Configuring Jenkins master and slave machines can be an exhausting effort which has to be repeated from time to time, in case that was some major break on the new slave has to be added. Although Jenkins has excellent GUI support, there is still a lot of manual work that cannot be fully automated and must be performed manually.

There is no actual guarantee that two different slaves will be the same when configured the same way. Even though using Docker inside Jenkins pipeline reduces uncertainty, there could still be unknowns in the configuration that can cause trouble at some point.

CircleCi pipeline, as already described, is fully described inside a single configuration file, while job executors descriptions (Dockerfiles and helper scripts) is also a simple configuration which can be quickly reviewed and modified.

Winner: CircleCi

Winner: CircleCi

 

Author Senad Zaimovic