As new technologies emerge, the way we work changes. So how might the AI-enabled support teams of the future function?
For the second video session of the Future-gazing AI series, our VP of Sales, Brónagh Crowley, dives into this topic — along with our friends from support quality management platform Klaus. Brónagh was joined by Klaus’ CEO and Co-founder, Martin Kõiva, and their Head of Machine Learning and Data, Mervi Sepp Rei. In this interview, our speakers discuss:
- How to bring rogue bots under control
- Whether humans will stay relevant in our fast-changing world
- Ways to distinguish between AI charlatans and companies that offer the real deal
- And much more
Find the full video session here, or you can read the transcript below.
Brónagh Crowley: Welcome to the Future-Gazing AI series. I’m Brónagh Crowley, joining you from London, and I’m VP of Sales at Ultimate. Today we’re very lucky to have a close partner of ours, Klaus, joining us. For those who don’t yet know, Klaus is an AI platform for quality management of large customer support teams.
Today with us, we have Martin Kõiva, CEO and Co-founder, and Mervi Sepp Rei, who’s Head of Machine Learning. Would love to have a short intro from both of you, if that’s okay. Martin, perhaps first?
Martin Kõiva: Yep. My name is Martin and I'm a Co-founder and CEO, mostly on the customer-facing side. Before Klaus I was the Global Head of Customer Support at a SaaS company called Pipedrive — so experiencing the problem firsthand that we’re solving now. And — important disclaimer — I have a degree in journalism. So anything that I say, take it with a grain of salt.
BC: Pleasure to have you with us. And Mervi?
Mervi Sepp Rei: I’m Mervi, I’m the Head of Machine Learning and Data. I’ve been with Klaus since the beginning of our data science efforts. I have a PhD in applied physics and I’ve done mathematical modeling — from how energy is transferred in the heart muscle cells, to how murderous waves are moving across the oceans, and how light moves inside a photonic crystal.
Since my academic career, I’ve transferred into doing text analytics in the customer center world. I’ve had my own startup, building models for prioritizing content based on text in different languages. And I also have experience doing business analytics in contact centers.
BC: Super impressive backgrounds — and definitely puts you in a strong position for what’s happening at the moment. And then obviously — with a background in journalism as well, Martin — I’m sure we can ignore GPT hitting non-stop headlines. Would love to know, if it’s not too trite, how has that impacted you immediately within Klaus.
MSR: I can say that it has significantly impacted us. It came to us at a perfect time.
We’ve been building these data processing NLP pipelines for two and a half years. And now when these ChatGPT models exploded — or in general, large language models (LLMs) became everywhere, like every week there’s a new model — our data processing pipelines were mature enough that we could immediately integrate these (not just ChatGPT and open AI models, but other LLMs) into our production pipelines. We could get AI products out to customers, which was amazing.
“Things that we thought we could do — but it would take us much more time before these large language models (LLMs) became available — now we can implement these things in a matter of weeks. So it has changed a lot.”
BC: Are there recent AI products that you’ve launched in the past few months?
MK: Excellent question.
MSR: Excellent question! Well, our GPT offering started with our CSAT survey revolution. It’s based on this notion that when people get these surveys, they maybe don’t remember what has happened.
It can take up to 8 days from when the conversation started to when they get the survey. So they don’t remember, and they might not be very well positioned to actually give adequate feedback. So we use GPT to generate summaries, to put them in the right frame of mind, to remind them what went on.
And because the most insightful information comes from when they write these free-text comments in the surveys — and it can be hard to start writing — we also (based on the conversations that they had) generate what the customer might want to comment on. They can, of course, redact and edit.
But this was our first Klaus GPT product. And recently we launched the first iteration of auto QA.
“When people do reviews, on average a mere 2% of the conversations get reviewed. And if you would compare, like an iPhone has approximately 100 square centimeters. and if you try to operate it with a 1.4 x 1.4 centimeter window — you’re clearly missing out.”
So what Klaus is doing now is that we are making it possible for you to see the whole of your iPhone — 100% of your iPhone — by automating the reviews.
MK: Maybe it’s worth starting from a little bit further away, because not everybody knows what Klaus is, or what we do. So the core of the product offering is what most large support teams call the QA process — by which teams internally review what has been said during customer support conversations.
So it’s the thing where you call an airline and they say, “This call is being monitored for quality purposes,” and Klaus offers a platform for doing that if you use a modern SasS help desk, like Zendesk for example. And it used to be fairly tedious, and you could only manually sample maybe a couple of percent [of interactions]. But now with the launch of auto QA, Klaus reviews 100% of your conversations for quality, automatically, out of the box.
It’s taken Mervi and the team years to get to — but that’s actually the reality now.
BC: So you really were at the perfect timing and the right position to launch this.
MSR: Yes. We started this auto QA thing a year ago, and then we knew that we would have to automate this category by category. We figure out how to do empathy evaluations, we figure out how to do tone evaluations. So this is how we progress. We figure out how to do grammar evaluations automatically.
So we knew what we had to do. And then suddenly — things that we thought we would have to label and train ourselves — all of the contextual knowledge of the written universe became available, so we could use that. And it’s also multilingual out of the box. So awesome, awesome timing for us.
BC: And how is this changing the industry right now? Because before, you were seeing a fraction of the iPhone, it’s really painful, you’re reviewing only a small percentage — seems like there was possibly a lot of guesswork about what was going on inside. So what are you seeing industry-wide for customer support, that is changing now as a result of this?
MK: The list of things that will change is extremely long, and we can't even imagine the exact things that will change. It feels a bit like the mid-nineties when the internet was getting its start — it’s like “Okay, this is clearly very, very big,” but nobody could have imagined what it leads to exactly.
Obviously everybody is fairly certain that the bottom whatever percent — 10% or probably more, 50% or 90% — of customer conversations will be solved by, maybe Ultimate, or some other bots that get smarter and smarter.
“But from where we stand, we still believe — maybe more so even than before — we believe that the value of human interactions is going to increase.”
It might decrease in number, but increase dramatically in value. So you don’t need to hire hundreds of agents to solve basic questions.
BC: So in that case, do you think the profile of people working in customer support is going to change quite dramatically? Especially if we’re putting human interaction as the premium and the most highly-valued, and a huge mass is going to be automated. Does this mean entirely new careers are springing up in its place?
MK: I think the basic qualities that are valuable in a customer support professional will remain the same. It’s just going to be the cream of the crop that will remain relevant. Human empathy will become the thing that has a premium. So I think probably no dramatic changes, but it will be those that are the best at what they do, who will remain relevant.
BC: I love that.
MSR: I think also, there will be wider adoption of these generative bots or virtual assistants all over the place. And where it leads us is that, this job that for some, maybe at the entry-level position — we tried to take these tedious parts out of it or the mundane things, so it will actually become more human thanks to these bots.
And there will be seamless integration of generative bots, traditional NLP bots, and agents. Then bots or virtual assistants can help humans, and the human can be the final person that then, you know, decides something or communicates with a customer.
But these tedious tasks of sorting through information, or checking something and digesting can be done for the person. So it makes it a bit more enjoyable to be an agent or a customer support representative.
BC: We’ve talked about the huge opportunities and advantages of working with AI. Are there any challenges that customer support leaders should be mindful of working with AI — and especially short term, while everyone is in this rush to use it?
MK: This is clearly for Mervi and I think, on the tech technical side, there’s a lot to talk about. I’ll say one thing on how companies position their offerings: The worst thing that you can do is try to pretend that whatever AI solution is now in place of a human, definitely — it will end badly if you implement any kind of AI solution and try to trick customers into thinking they’re talking to an agent. That’s not gonna end well.
MSR: Yeah, definitely. This is maybe far-fetched, or a little bit too much in the future, but if we are talking about building very specialized models or models that do specific tasks that the general models everyone has now plugged into don’t solve very well — to build these models, you have to have very trained, very skilled people.
“So we might be reaching this barrier where we cannot train the models to be smarter, because there are no people that know better.”
And another thing is — how to have common sense around what you should let the bot do, and what you shouldn’t let it do. You shouldn’t try to replace humans, like Martin was saying. I fundamentally agree. And you shouldn’t fake it. But there are so many tedious, boring, error-prone things that people do — that are a little bit cruel to make people do. So the bots can definitely take over these things.
Another area that is problematic is: Businesses evolve. So how to monitor that the bot is doing the correct thing? How to stop it when you see that it’s going rogue? How do you see that it’s going rogue? So how to make sure it’s continuously getting better is interesting. And Ultimate is probably facing the same problems. So making these [technologies] evolve in a transparent way is interesting.
MK: Yeah. This is probably pretty appealing to many companies: We’ll implement AI, it will take 70% of the easy cases, problem solved. And even if they do a good job, one thing that companies are not necessarily thinking about is:
“The other advantage of having humans doing customer support is that you generate constant operational awareness of what your customers are telling you.”
If you lose that all of a sudden, it means you’re not necessarily getting the signals to Product [teams], for example, or if something else is going on.
The bot is there to solve the questions, but the bot may not be programmed to communicate about trends that are not related to the questions being asked. Maybe there’s some issue coming up, and that’s where Klaus comes in. We can facilitate bot reviews as well — and that’s gonna be even more important going forward.
BC: I would agree, and I think it’s a really good point as well about going rogue. We hear people being like: “Well, there’s GPT, we can do this for ourselves.” But we were so careful in how we built our new GPT product, and it was so important that we still have those measures of control, because ChatGPT can literally fling anything at you.
And what else was so important to our customers was that we tested slowly. We started with our existing [customers] first and approached it in a really controlled manner. It was linked with your knowledge base, you knew exactly what data the AI had access to, because — as you mentioned — otherwise without those controls, anything could happen. And you don’t have those human elements to alert you and to guide you — and also to escalate when you do need a human involved as you go through.
So I think especially at the moment, it’s more and more apparent as so many new companies are coming on board — which maybe didn’t exist six months ago, and now there’s this sudden rush. I think it kind of hones in — and it was another huge reason why we were so happy to partner with you, because we’re partnering with years of experience and knowledge and intense proficiency in this area, which I think for customers and your brand integrity is so, so important.
MK: For those that are on the consumer side of this technology, it’s going to be very difficult to distinguish between providers that know what they’re doing and those that do not. Because everybody is slapping the AI label and everything. A colleague recently said that they went to a conference and it seemed like that was a prerequisite for being at the conference, that you have AI somewhere in your name.
“One common sense thing to keep an eye on is how long a company has actually been engaged in some kind of data science related endeavors. And also simply: Do they employ data scientists — which is often a way to cut through the noise.”
I know that in our market there are quite a few [companies] that claim to do AI-related things, but then if you go on LinkedIn, then you’ll see: Well, but who?
MSR: And another area is that everyone is doing some NLP or large language model stuff, but you can do a lot of things with other kinds of statistical modeling — even just to prepare the [data] sample or something like that. So using this information together with LLMs is truly, truly powerful. But you have to be smart about it.
BC: Absolutely. And it sounds like there’s a lot of activity and speed moving through your product. What are you most excited for? What’s next? I can see you already have releases within a matter of a few months. What’s going to come in the next year for Klaus?
MK: The general promise of 100% coverage that works out of the box, that doesn’t require sophisticated implementation: That’s going to be the main focus in the short and midterm.
Improving on the feedback-related offerings — that’s gonna continue to be another focus, where we help you get more text-based feedback from your CSAT surveys. So I think those would be the two things I highlight. But Mervi, maybe you want to add more.
MSR: I can echo that or repeat that. We will continue working on this auto QA offering, making the coverage wider in terms of what we understand and what we can do automatically.
And because we are in this 100% range, and we are more and more operating together with humans and bots, then making this information available and trying to have optics into understanding how your bots are doing. Not just how your agents are doing, but also how your bots are doing, meaning better analytics.
MK: Yeah, and I think in this economic climate, the good news is that this is a great way — just like Ultimate — it’s a great way to do more with less. Because there aren’t too many companies out there that have budgets that are increasing. So this is a way to get more out of the resources that you have, while actually getting higher quality, orders of magnitude, like better insights and coverage of support quality reviews.
BC: Well, thank you so much. Final question to wrap up, and perhaps I’ll give it to you first Mervi, which is: If you had to sum up your opinion on the future of AI in 3 words, what would they be and why?
MSR: I would say empowering, transformative, and evolving. AI has potential to empower individuals, as I said before, it can help people to be more human — doing things that humans are good at. So automating mundane tasks can empower people, augmenting human capabilities and enabling them to be better humans in a way.
“It is transformative. It can do things that were, if not impossible, incredibly hard just half a year ago or a year ago. So plugging this into customer service has huge, huge potential.”
And since this thing [generative AI] exploded, there are so many more capabilities that have come out already. Like you can analyze text and images together in the same model, and do different plugins so that you can be not just a very smart answering thing, but you can also build better connections with the backend so you can actually do something when you are certain enough that it’s a valid action to be taken.
BC: Love that. And Martin, would you agree, or anything additionally to add?
MK: Definitely. Well, my first answer would be ‘we shall see’. Because nobody really knows. But I could also say ‘super high leverage’ because I think — it’s very similar to Mervi’s answer actually. I think we’re gonna see companies and customers doing the same things that they’ve always been doing, but with 10 times, 100 times the resources.
“Things that weren’t possible, simply because of technological barriers, we will now be doing with large language models. But that’s true in all categories, not just in customer support.”
BC: Well, we shall see. Anyway, thank you both so much for joining us today. Like I said, very proud to be partnering together. Thank you so much for your time — our second guests ever on our Future-Gazing AI series. So thank you, have a lovely evening, and talk to you all soon.
MK: Thank you.
MSR: Thank you for inviting us. It was fun!