All Hands on DevOps #5 – Christmas Edition

Jason Man 15th December 2016

On Tuesday, we hosted our 5th meetup of the year – and had a massively successful turnout given how close we are getting to Christmas.

Canapes.png

The event was hosted at the amazing Smarkets offices.

We  were greeted with drinks and canapés on arrival, from Smarkets’ very own chefs, which I have to admit, were very nice!

Now, down to the real business of the meetup.

We had two great speakers lined up for the event:

  • Matt Smith from Astillion
  • Stijn Polfliet from Coscale

For those that were unable to join us, below is a quick summary of what we covered on the evening. We’ve left out the Q&A though – you’ll have to attend one to see exactly how we question our speakers!

Matt Smith @soimafreak

CTO at Astillion 

Matt talked about how openshift can be used cross-platform with PaaS on a cloud infrastructure.

It allows you to reuse most of the components (ssls etc…) to a different type of infrastructure (on prem, virtualised, cloud). This, in turn, allows your organisation to scale as you grow, and adopt new technologies without needing to rewrite or reimplement your infrastructure solutions.

This enables you to quickly build using Terraform, as you can move to a module provider mechanism. This is best followed using the Hashicorp best practices page on Github.

By moving to the module provider mechanism, you are able to manage different environments easily without any major changes to the code. When creating the Terraform images, you can create tags that tell Ansible exactly how to build the infrastructure with the exact setup specified for the right environment. 

Picture1-6.png

Scaling with Terraform 

This is as simple as changing the count in Terraform from X to Y. It then simply builds the infrastructure as required. 

Pros and Cons

Working with openshift helped Matt discover many pros and cons with working with these set of tools, some of these include: 

Pros Cons
Terraform
  • Terraform works great on AWS 
  • Easy to use when you know how
  • Provides some abstraction for more complex infrastructure
  • Poor documentation 
  • Unable to keep up with the pace of IaaS providers 
  • Can become messy when you have multiple products and multiple environments per product. 
Ansible
  • easy to use low barrier of adoption
  • lacks difference engine 
  • A puppet hiera data equivalent would be great! 
Openshift
  • easy to get up and working
  • Great for “getting stuff done” 
  • Good support
  • Lots of administrative tools
  • massively complex under the covers 
  • Steep learning curve, maybe in three months you will be good
  • Has “quirks”

Stijn Polfliet @spolfliet

Director and Co-Founder of Coscale

Stijn talked about performance management for microservices.

Whilst containers are great, and can really scale up – people are still involved. Because of this, there can be some bumps along the road when adopting microservices and containers. The talk was focused around how we can understand the performance of containers and the different metrics involved to do so.  

Monitoring

Monitoring is an important aspect to any successful implementation, as we want to move towards a more proactive than reactive model. There is already: system infrastructure monitoring, orchestration monitoring, container resource monitoring, incontainer/application monitoring and finally real user monitoring.

Each one of these monitoring systems offer valuable information to the end user. 

Challenges

We see when monitoring microservices are Scale and Dynamics, the number of containers used grows exponentially vs. the traditional method of managing infrastructure (build and destroy) and this is done at a much faster pace.

On top of this, there is a large diversity of the different tools available to be used, which means there are multiple combinations and variables to consider when deciding what information can be gathered. 

Typical metrics gathered on traditional application landscapes:

  • On a OS level there are 100 metrics and then on a application it’s 50. This is already 150 different metrics to gather. 
  • Now in microservices landscape, at OS 100 metrics, orchestrators 50 metrics, per container 50 metrics,  per application 50 metrics. That’s almost 1000+ per host for the number of metricsthat can be gathered!

Some tools available to get information today: 

  • Docker stats api 
  • cAdvisor 
  • APM vendors: New Relic, Ruxit, and Appdynamics 
  • OSS: Prometheus, Graphite and Nagios

Anomaly detections

Learns the normal behaviour of your system and when something goes wrong it will alert you. 

Having static alerts will have limitations as it cannot account for seasonality, correlations of data, and changing or dynamic environments. 

Coscale 

A full stack monitoring platform, uses a lightweight agent for production use. It can do anomaly detection and understand events in context. It’s easy to use and easy to get going with options for SaaS or on premise. 

Smarkets.png

As you can see, the evening covered some really insightful topics – as well as a fierce Q&A session afterwards for both our speakers.  We’re sad to say that this was the final meetup this year – but we have some amazing venues and speakers lined up for everyone next year.

Until next time, Merry Christmas and a Happy New Year from everyone at ECS Digital & Third Republic.  

If you’re interested in speaking at, or hosting one of our next All Hands on DevOps meetups at your venue, please do get in touch

Found this interesting? Why not share it: