Jenkins: The issues around selecting, not investing, in a tool

Jason Man 2nd June 2017

Over recent years, Jenkins has built a reputation as a very reliable scheduling build system. With 1360 plugins at the time of this writing, it’s currently one of the most popular build automation tools. However, its increasing popularity has seen a broader range of teams, with varying degrees of technical skill, adopting Jenkins as their driver to Continuous Delivery.

Due to their lack of technical knowledge, these teams require a more bespoke user interface. Enter Blue Ocean, a plugin that introduces a new user interface that makes Continuous Delivery more accessible to this new audience, without sacrificing any of the power of Jenkins.

However, Jenkins is used very differently in each organisation and many businesses find that they are unable to get the results they expect Jenkins to deliver.

Identifying the challenges

ECS Digital has implemented and optimised Jenkins in many leading organisations. In our experience, issues appear when businesses fail to invest in the processes that enable them to get the most out of the tool.

“The tool selection process can be long and protracted. So, when a business settles on a tool, the temptation is to begin using it as soon as possible” says ECS Digital Founder and Managing Director, Andy Cureton.

By neglecting to invest in the infrastructure to support Jenkins and failing to implement best practices for usage and training, businesses are diminishing their chances of successful implementation.

Examples of challenges that businesses could face when selecting to implement Jenkins rather than investing in it as whole are as follows:

  • Updating and upgrading – Challenge when doing routine updates and also major ones, like migrating from Jenkins 1.x to Jenkins 2.0. A common issue being the reliance on many plugins, if a plugin isn’t compatible with the new version it might break entire pipelines.
  • Scaling – Issues with scaling or availability. A master with 4 build agents might cope with 100 builds a day, but might not be adequate for 1000 builds a day.
  • Pipelines – Challenges with implementing pipeline as code. It is easy to overload pipelines which will put all the load on the masters.
  • Integrations – Connecting other tools and services in a way that matches company policies, and enables controlled access to sensitive environments.
  • Performance – Hitting an internal road block or not performing to achieve the long-term business goals.
  • Personalisation – How to build a CI pipeline for your system.

Fixing the problem

Making sure that your implementation is architected to support your business from the outset is a key way to avoid the above issues. Here are two real examples that illustrate not only the businesses challenges, but the solution we have implemented to overcome them:

Example 1: We were engaged to help alleviate the growing pains of a Developer Services Solutions team within a global, UK based financial institution. Due to a massive growth in the amount of Jenkins users, the team struggled to deliver more Masters and Slaves, a manual process, while still working on day-to-day tasks.

To solve this problem, we automated the installation and configuration of CloudBees Jenkins Masters and Slaves, so that the Jenkins service could be built end-to-end in a matter of minutes. This meant that new teams and users could be on-boarded much quicker, while also ensuring the same configuration would be used on the entire Jenkins estate. This delivered a more stable and predictable environment for the consumers and the maintainers of the service.

Example 2: We were engaged by another leading UK financial institution to assist with an issue around the creation of jobs in Jenkins. Upon arrival, we found a monolithic Jenkins instance running around 6,000 jobs. After closer investigation, we discovered that over 5,000 of these had been created by a single individual, an issue caused by a lack of understanding of up-to-date best practice use of features and processes.

To solve the issue, we identified configuration and installation practices that were outdated, and introduced a new roadmap to the team, showcasing best practices and new processes to follow. We also introduced a new feature to assist with increasing the workflow output. This allowed the tech team lead to create seven templated jobs replacing the 5,000+ jobs created from the 2 initial seed jobs.

Coming to a conclusion

You get out of a tool what you put into it so it’s important to take the time to secure the processes around Jenkins. As Andy Cureton says, a process that works for 10 people may also work for 100, start creaking for 1,000 and completely fail for 10,000”.

As part of our Jenkins Health check service, we spend time on site to understand your system to help you fix and optimise Jenkins, so you can get the results you expected.

Enjoyed reading this? Get in touch with our team today.

Found this interesting? Why not share it: