Leveraging Amazon Connect and Lex to gain insight from your customers (Part 1)

Tony Connor 14th March 2019

In this 3 part blog series, we’ll be discussing Amazon Connect, Lambda, Lex, Elasticsearch and Kibana and how they can be used to quickly gain insight into customer activity. Amazon Lex is a great service and for conventional usage, where you know what to expect a customer to say, it’s great for automating and providing self service to customers. Our involvement with Lex has been on the backend of the Amazon Connect solution that we have been implementing with customers. One of the really important parts to a transformation is understanding your baseline. That doesn’t mean working out how we do a ‘like for like’ or ‘lift and shift’ from traditional black box solutions to Amazon Connect, it means we need to know what our client’s customers actually want to talk about.

This brings us on to a pretty unconventional way to use Lex, rather than assuming what the customer will say (utterances), we use it to record an answer after asking the customer what they want, we can then use the results and insights to design better interactions. Another reason for the way in which we’re implementing it, is so that we can conduct ‘free speech’ surveys as part of a Contact Flow. This data eventually leads to better categorisation and routing of calls, benefitting the customer by improving first call resolution rates, lower wait times and organisational insight.

If you haven’t yet configured an Amazon Connect Contact Centre, you can visit the following blog How to stand up a Contact Centre in 15 minutes

Let’s look at an example of how you might implement this in the Contact Flow, and then we can discuss the various components:


Let’s start at the beginning and give an explanation for each block:

Entry Point

This is where the contact will enter the platform, every Contact Flow starts with one of these. It is always the first block and is used to join to the rest of the Flow logic.

Set Logging Behaviour

This block is essential to the solution, when it is configured in this way, do not forget to add it. If you don’t add and enable logging on the Contact Flow, the data will not be delivered to Contact Flow Logs in CloudWatch Logs.

Play Prompt

In the context of this Call Flow, the block plays the question to the customer. This can be any question, ultimately what we’re trying to do is capture why the customer has called (the data for which will be used later to construct a better model for contact routing. In most cases, when you use a Lex Bot in a contact flow, you will have some kind of idea of the answer (or utterance) that the customer will respond with. In this case however, we could get any answer, which means triggering the Bot isn’t as simple as you might think. Because of that, we need to trigger the Bot into a listening state, prior to the customer responding to the question, that is where we get to the Lambda.

Invoke AWS Lambda Function

The contact has just heard the question, we now need to trigger the Bot into a listening state so that when the customer responds to the question they have been asked, we can capture the answer. The code to do this is as shown:

Here we are using the Python Boto3 SDK inside the Lambda to call the Lex Bot in this case called CustomerSurvey (you can see this in the next block in the ContactFlow). We use a random string of characters, one the customer will not say, to trigger the Bot into a listening state, ultimately this is a session and the ID will be the ContactId we send through as userId.

This all happens on average in about 80ms, which is important, because if there was any delay here, we wouldn’t catch the customers entire answer. Note in the Lambda we create connection to the lex-runtime API. Note whilst it’s not clear, the Lambda has a role that enables it to call the API. Within Amazon Connect, the response is expected to be a Key:Value pair, rather than an error, so in this case we send back a success message.

Get Customer Input

The Lex configuration for this Bot is very simple, we are not using Initialisation & Validation or Fulfilment Lambdas and only use the ability to record the customer and automatically transcribe the response. You can of course have more Slots and provide a sequence of questions, using Lex to control the flow, however in this simple case we are only looking for the single answer. The configuration for the Bot looks as follows:

As explained, there is a single Intent, which is triggered by the configuration in the Contact Flow ‘Get Customer Input’ block as shown below:

Because the Lambda has already triggered the Bot, it is already listening, or ready to accept another event, in this case it is the same ContactId (userId in the Lambda call) that will speak, causing the answer to be recorded in the {Question}slot. This is the slot that is then returned to Amazon Connect in the $.Lex.Slots.<attributes>section.

To prevent you needing a prompt on the Lex Bot, you will need to configure a null value in the Text-to-Speech section as follows:

This allows you to pass the customer straight through from the Play Prompt block to the ‘giving of the answer’ by the customer. We have tested this significantly and as noted already, once there is a steady stream of customers we see <80ms on average. Be aware that if you have very low call volumes, you will see delays due to the time it takes for Lambdas to ‘warm up’.

Set Contact Attributes

There are a number of ways to get the value of the answer the customer gives. You could use a Fulfilment Lambda inside the Lex Bot to update the Contact Attributes via the Connect API. You could even pass through the values through a Kinesis Stream that would be subscribed to the Contact Trace Records. In this instance however, we are showing how to add them to the Contact Attributes by the Set Contact Attributes block in the Contact Flow. This means that the data will be available in the Contact Flow Logs, which are kept in CloudWatch.

We set an attribute called Answer to $.Lex.Slots.Question. This stores the response from the customer to the question we are asked. Note that the Question segment of the variable is the same as the Slot Name in the Lex Bot (shown below). In a subsequent blog, we will talk about how you can extract this information to create dashboards and insights using products such as Elasticsearch and Kibana.

The configuration can be seen below:

Other Blocks and Concepts

We aren’t going to cover Error States in the blocks, or branches/actions based on those states, for example the transfers here will fail, they are dummy transfers. It does depend on what you want to use this for. In cases we have used this as a solution to ask customers what they are calling for, we pass them to the original queue they would have called, since at the moment we are just carrying out the first stage, investigating their needs and leveraging the results to build better services.

We also haven’t gone into the detail on the process we use for configuration and deployments. ECS use Infrastructure-as-Code (IaC) wherever possible, where not we will use Custom Resources in CloudFormation and finally if we cannot do this due to no API availability (such as with some parts of the Contact Centre) we will manually create items.

What’s the outcome?

We’ll be doing another blog that will go into more detail on how you can visualise the data that is being recorded as part of this contact flow. Ultimately the immediate outcome for a business might not be obvious since we haven’t created any dashboards. However, you can see below that we now have an event in our CloudWatch Logs for the Amazon Connect Instance (note, CloudWatch Logs should be turned on). A log looks as follows:

Above is a screen grab of the ContactFlow log, you can see that in the Parameters segment we now have a Key Value pair that contain Answer (which we set in the Set Contact Attributes block) and the Value. Using the solution mentioned earlier for Elasticsearch/Kibana, you can index and extract these values and use things such as Word Cloud to display the most commonly spoken answers. When you scale this and have many contacts and answers, the data set can be used to build models and inform decisions around how to construct better Lex Bots.

In part 2, we will look at how to transport this data from CloudWatch Logs over to Elasticsearch in preparation for creating dashboards in Kibana.

Found this interesting? Why not share it: