Key learnings from Vodafone and Nationwide at DOES21

Olwen Davies 1st June 2021

When I joined ECS digital two years ago, a huge draw was the opportunity for technical growth made available to consultants.

ECS enables ongoing learning by aiming to have a utilisation or billable rate which ensures all consultants get time away from client work to study, spike new initiatives and attend conferences.

Another avenue for ongoing learning is our work with partners who provide us with early access to new developments, training, and support channels for the use of their products.

CloudBees is one such partner.

As a service provider for CloudBees, ECS is recommended to clients and pick up projects at whatever stage is required to implement solutions using tool best practices to optimise the client benefits.

As consultants, we equip ourselves with available partner training then reinforce and consolidate this through hands-on use cases or internal projects work. For example, after using pipelines many times on client site, I followed CloudBees online training. This training was reinforced by setting up a CloudBees Jenkins CI pipeline to build a node JS API & database within a Docker image, whenever I made changes to the code.

Back to the partnership…

As part of a CloudBees and AWS-led project, ECS was introduced to a multi-national telecomms client. Personally, this partnership gave a great opportunity to work in a team on a green field project.

The remit was to improve the deployments of a third party delivered application, reducing manual input via automation and standardising to give a single implementation and documented approach across countries. CloudBees CI & CD were configured and integrated with other tools within a private cloud. This helped to manage multi-site releases of a multi-tiered, internally hosted mobile payment application. Additional CloudBees tools were also leveraged to combine and orchestrate all the different stages in one easy to use interface.

 

How did we work together?

The mind map below illustrates the service provider “doer” to partner relationship.

CloudBees provides tool training, R&D recommendations and gives us the bigger picture context of what is happening in the enterprise tech world. This prevents us living in single projects or organisation bubbles.

ECS, on the other hand, leverages its experience as a service provider to assist clients in the practical use of the partner product in whatever way is needed. In the words of Martin Roxby, Director of Professional Services in EMEA for CloudBees:

“ECS has done a great deal for us, picking up work at scale which we can’t do as a software vendor …it’s great to have a partner who can fill in the gaps, proper experts in areas our clients need support on.”

Our support comes in all shapes and sizes. For example:

  • Design and definition,
  • Implementation,
  • Work within an organisation to actualise a project whilst enabling staff in a handover and skills share,
  • Provide a team to resource new developments.

This partnership also hugely benefits from our mutual interest in knowledge sharing.  I was incredibly lucky to have been invited by CloudBees to attend the DevOps Enterprise Summit 2021 (DOES21) – an annual conference for the technology leaders of large, complex organisations implementing DevOps principles and practices.

 

Here’s some of what I learnt from DOES21…

Over time – and with experience of varied projects – engineers realise that HOW you define work, understand the context of the project, and communicate within the organisation, can be as critical to project success as the WHAT of the technical solution or WHICH tool you implement.

A risk inherent in DevOps is that complexities solved by code may lose transparency to the customer.

A tool like Cloudbees CD intentionally abstracts away complexities to execute the requirements behind the scenes of a single interface. It simplifies the tasks for the customer or support users.

If a customer is no longer connected with the “whys” and “hows” of the work, they may tune out through a lack of confidence or lose sight of what the problem being solved was in the first place – and thus how great the solution provided is.

When you have engineers doing clever stuff, how can you keep the stakeholders understanding what is being delivered, the difficulties encountered and the value therein?

My CloudBees ticket to DOES21 was an ideal chance to consider the conundrum inherent in taking on customers’ work and keeping them connected.

Amidst a deluge of information on large scale cross team initiatives in banking, telecoms, and the shared platform public service giant Government Digital Services, I kept asking…

“How do you keep a customer engaged and carry them along with highly technical work?”

“How do you position DevOps as the key enabler in both compliance and fast delivery?”

“How do you ensure the customer appreciates the value of the technical work delivered?”

I focussed on enterprise level “war stories” hoping to find common themes bedded into organisation approaches.

In each case, DevOps programmes were directed towards clear business goals. This was achieved through value stream mapping or a thorough breakdown of requirements to ensure that by clearly understanding outcomes, the right thing was prioritised.

A clear theme was that DevOps customers are notoriously opinionated. In my experience, I have found that Software Engineering and Support teams need to be carried along, have a desire for change and to learn to own and manage as well as use the pipelines. I have also seen Audit teams who provide oversight and accountability for compliance and need convincing that automated reports can be transparent.

What was refreshing about DOES21 was that I wasn’t the only one to be asking these questions or spotting these patterns.

Keynotes at DOES21

CloudBees hosted Nationwide to discuss “The epic balance of moving faster and safer whilst reducing risk”. This talk was particularly interesting because it flipped the narrative that ‘technology drives transformation’ on its head.

Ben Angel, Head of DevOps engineering at Nationwide, described the bedrock of a successful DevOps implementation as “People, process, tools”. IN THAT ORDER!!

“Tools are critical to us, but they underpin the people, the cultural changes that we normally need to make as well…. We need the right tools for the job and what we’ve done is we’ve worked very hard to align the scale of our tools, the build of our tools and deployments of them with the people and cultural methodology training alongside the ongoing help and oversights that we know we need as well.” 

In fact, the talk explored Nationwide’s own engineering platform team, showing that CI/CD is directly linked to business objectives to enable them to be “more efficient, to embed deeper controls to adopt DevOps to give better service to members”.

For Nationwide, the remit is “to build the right CI/CD solutions to enable engineers to deliver safe controlled pipelines. It is a framework to ensure that going fast does not mean sacrificing compliance”.

 

Top Tips from the Talk:

> Nationwide accredit some success to being clear of how to measure efficiency before you start, so you can describe the benefit as part of the transformation.

> Define MVP:

  •  Ask what can you do to get demonstrable value as quickly as possible? In the same breadth, create understanding in stakeholders.
  •  Adopt three identifiable measures:

1. Release velocity / time savings on key deliverables:

  • Infra as Code environment builds
  • Test cycles
  • Automated reporting

2. Source code management maturity

3. Time and cost reduction of multiple environment provision

> Remember a tool is only as good as what you do with it. Stick with the basics. Keep it simple, focus on the organisation product needs out the box implementation of extra tool functionality.

> Take on a single tool and eradicate duplication.

 

The Benefits Nationwide have seen:

> An Iterative approach gets features to customers faster with feedback loops:

  • Deployment via CD – allows flexibility across release approaches (Blue/ green / canary)

> Reusable templates save time across the organisation.

Customers of the DevOps Pipeline are development teams who then use this template knowing that safety, security, and compliance are covered for regulatory requirements. This helps to:

  • Enforce Pull Request compliance
  • Scan for security risks
  • Embed those controls into gates prior to progression to next stage
  • Ensure fast pace deployment by moving away from multiple assessments.
  • Cover compliance as tests and regulatory controls are built into the pipeline via Intelligent controls, with a single reporting mechanism.

Tangible results:

  • People are engaged in DevOps. Trained engineers are technically proficient and bound to agile/DevOps driven ways of working. Ensure there is a feedback loop, and that central resource has accountability to act on input from teams.
  • The Auditing team are happy with the automated compliance key controls and reporting I.e. The result of building quality checks, data archiving, GDPR regulatory requirements into the pipeline.
  • Security can perform threat models on pipeline access from source code to production. Issue regular reports of threats and mitigation.

In the words of a Nationwide Software Engineer:

A clear result was more safety achieved than in previous releases. Engineers have more time for improvements, less manual change, and less downtime. The automation of regular standard tasks has freed up time for innovation, thus supporting an agenda of technical advance and continuous improvement.”

 

Transformation is a people’s game

The notion that people and their requirements take priority over being led strictly by technical drivers merits more exploration.

I was struck by a Vodafone presentation which again emphasised ‘people’ factors as critical to CI/CD long term success. This time exploring the setting of targets, their communication and moving in the right directions by gaining participation through ensuring the goals are shared and understood.

In his breakout session “Driving Cultural Revolution With OKRs at Vodafone UK”, Ben Connolly (Head of Digital Engineering) describes himself as “constantly learning about new ways of working”. He spoke alongside Sabina Kamber Salamanca (Lead Agile Coach).

Together, they set out a new approach of augmenting inhouse technical capacity to work with and learn from partners. A change from outsourcing to third parties and then waiting to receive and check deliverables.

For Vodafone, its own software engineering team has doubled in 18 months and is at the heart of what is delivered to customers “connecting the world for a better future”. Vodafone feel that it must be a product-led company, driven by putting technology first. In fact, they predict that due to technical growth, 40% of its staff will be involved in software engineering by 2025.

Ben Connolly advises anyone seeking change on a grand scale to “recognise it is a series of micro transformations, small steps which culminate into a transformation”.

An “Objective Key Results”, henceforth OKR, approach has been used to encourage relevant and timely initiatives with teams engaged and feeling empowered to identify and solve problems.

 

Previously, Vodafone was working to over 100 KPIs (key performance indicators) which encouraged gaming and repetition of the same habits, at the expense of real value for customers. It now seeks to raise awareness that “every release is an opportunity to stop theorizing about what customers want and actually put hypotheses to the test, getting our product in their hands”.

Most Digital Engineering teams now share the same overall objectives to encourage a move towards CI/CD and address the problems of legacy integrated platform ‘big bang’ releases which are prone to errors and delays.

1 – Every team must release to production at least once a sprint

Every team member also has a task

2 – Each person should lead an independent release at least once

If anything, Vodafone unquestionably proved that mindset change happens through actions. Within 6 months, 75% of teams were achieving the OKRs. The break away from monolithic releases is a status quo today, with all digital engineering teams releasing to production in small increments and independently. The average cycle time reduced from 30 days to <5 days, with features frequently being shipped early.

They attribute this to OKR Characteristics:

Closing remarks:

It was a refreshing to see the emphasis across talks at DOES21 on both people factors and how technical work must always be linked back to customer gain and embedded within the client teams.

We can agree that technology makes things possible, but we must remember to keep it simple, to ensure the things we do are widely understood. Ultimately, we all want to deliver a long-life , flexible and well understood product – to know that the work we do and the tools we use genuinely answer a customer need and can become ingrained into their ways of working.

Thanks again to CloudBees for letting me be a part of DOES21. I certainly have plenty more stimulus for ongoing learning now and more context for future client work.

 

***

About the author:

Olwen (she/her) is an ever learning Delivery Consultant at ECS. Technology allowed her to work full time whilst fulfilling single parent responsibilities, albeit with creative childcare solutions. She is a ‘Tech Talent Charter’ ambassador, promoting inclusion and the importance of diverse experience in product development.

Found this interesting? Why not share it: