Episode Transcript
[00:00:22] Speaker A: Hello and welcome everybody, and thank you for tuning in to Metrosite, a podcast for Metrogy. I'm Beth Schultz, Mutterjee's vice president of research and a principal analyst.
[00:00:34] Speaker B: And I'm Robin Garris, CEO and principal analyst for customer Experience.
[00:00:38] Speaker A: Today we are super excited to be talking about our CX Metrostar award. We have winners across twelve customer experience product areas. But before we share any more information on who our winners are, let me first explain a little bit about this award program.
First of all, it starts with our annual global buy side Metrocast study we do with CX and IT leaders.
We talk to them about their adoption and spending plans, their plans to change providers, when and why. We ask them to rate their providers on a number of criteria such as performance, platform reliability, technical capabilities, value, sort of bang for the buck. And then we ask them to share business success metrics. We use those latter two factors so customer sentiment ratings and business success metrics to determine our metrostar winners.
We designate three types of winners. So we have our Metrostar top providers, and with top providers we recognize those vendors whose customers not only achieved high business success, but also achieved at or above average customer sentiment ratings. And then we have top business success. And those are the providers that received at or above the average percentage of customers achieving business success and then top customer sentiment. And as you can imagine, that highlight providers with at or above average customer sentiment scores.
[00:02:15] Speaker B: Yeah, and you know, it's our biggest study of the year. We had just under 1600 participants in this study. So it's a global study.
We spent a lot of time gathering the data itself and also spending a lot of time on our data integrity methodology. So this is something we spend weeks on, just getting data in, looking at the data and cutting data that we can tell through a variety of our integrity cleansing methodologies that we use are just either bots or they're not answering, they're just clicking buttons. So our goal here in the data gathering stage is that we are getting people who are paying attention. They're reading the questions they're answering accurately.
We don't want bots. We don't want, you know, we don't want people who aren't paying attention. And we do a really good job, I have to say, of finding those and eliminating them from our total data set. So that's why it takes a little while, it takes us several weeks actually to gather our final data set where we are confident that the people who are answering are again paying attention, qualified to answer and so on. So that's one thing I want to make sure really stress is how much time we spend on gathering this data. And I think that's really important when you look at things like the customer sentiment methodology. I know other companies, when they're looking at other research firms, when they look at customer sentiment, might be using third party rating sites or other sites where they're not really able to validate. Oftentimes the customers or the IT or CX leaders are who they say they are. We do this all ourselves internally. We don't rely on third parties. I think that that's really important in terms of we don't rely on third parties to cleanse our data to make sure that it's all accurate. So I think that's a really important thing. I want to stress that now when we gather our customer sentiment ratings, as Beth mentioned, we ask them usually about eight or so questions about their relationship, about their sentiment for their primary vendor in all the different areas that we cover. So we asked them to rate them on a one to ten scale. We did change our scale this year compared to previous years, in previous years where we were using a one to four, which changed it to one to ten, just to give a broader range of scores and ratings that companies could give to their vendors. So that's one change we made this year. But a lot of the ratings themselves, like the topics that we're asking them to rate about, are pretty consistent year over year.
We did also change some of our business success ratings this year. So in years past, we always looked at four areas, basically, how did the use of technology affect revenue, customer ratings, agent efficiency and operational costs. And what we found with some of these more emerging technologies that we're now asking companies to rate is that they may not boil up their business success to some of these higher level areas. Like, you know, maybe they don't know how much, you know, bots increase their revenue. Maybe they do, but maybe they don't, you know, so we wanted to get a little bit more granular in the business success metrics that we were using. So we do still ask about revenue where it's appropriate. We do still ask about efficiency and cost where it's appropriate. But we also ask about more granular KPI's, things like average handle time, things like age and attrition that we know that they're measuring in relationship to a particular technology.
In years past, we always use those four that I mentioned this year. Each one of our categories has a different set of business success metrics that we're using, and we do show what those are in the report. So basically, we look at every vendor, we look at every it or CX letter that responded and participated in our research. We correlate their scores with the vendor that they rated their scores and their business success, and then we figure out the average business success number. And then for each vendor, we just look at what percentage of your customers rated you higher than average in something like revenue generated or average handle time. Where are you better than average? And then we crunch some numbers and do some methodology where we look at the total number of successful customers that each vendor has, and those who are above average get an award, basically. So it's looking at the percentage of your customers who are showing higher than average business success using your technology. That's basically the way we look at the business success number.
So the categories that we have this year, no particular order at all, but it's CCaas, on premises, contact center solutions or platforms, CPaas, so, communications platform, as a service, CRM agent, assist bots, workforce optimization, workflow automation and visual engagement, voice of the customer and self service knowledge management, and then wrapping up with AI development platforms. So those are all the categories that we have put in our metrostars for it this year.
[00:07:38] Speaker A: So obviously, we put a lot of rigor into this Metrostar program.
I'm excited to say that this year, after going through all of that, we determined 112 Metrostar award winners for customer experience this year.
That is 51 vendors who received top provider honor, plus 32 vendors who earned designation for having top customer sentiment, and then an additional 29 who earned honor for having top business success.
Okay, so let's go on to the winners. Robin, why don't you share the first few categories with us?
[00:08:22] Speaker B: Yeah, absolutely. So in ccaas, obviously a very important area these days, sort of the platform for all customer communications. The top providers are Alveria, Cisco, 59, and Nice.
The average customer sentiment score was an 8.13. And again, that's a one to ten scale. And 56% of customers on average, fell into the success group. Customer sentiment only awards went to eight by eight. Vonage and Zoom and business success only awards went to Avaya, Genesis, Sprinklr and Zendesk.
And just one more little piece of data. Five nine actually won the top, I should say one. But they had the top score for customer sentiment, and Alveria had the highest percentage of their customers in the business success category. So that's CCaaS on Prem contact center, which is also an important area in CX. We still see a lot of vendors even though a lot of the discussion and the innovation is happening in the CCaAS arena, we see a lot of big, big contact centers continuing to be on premises. So the top providers for on premises contact centers are Avaya, Genesis and Nice. An average customer sentiment score of 7.78. An average of 52% of companies in the success, the business success group. For customer sentiment only, we have Cisco, Nchouse and Zendesk. And for business success only, we have Alveria. Now, Avaia actually scored highest for both sentiment and business success when it came to on premises contact center. I will say look, overall, when you looked at all these different areas that we tracked this year on Prem, had the lowest overall sentiment scores of the study. But customers are, companies are still seeing business success in this category right now. So I mean, it's an older category we have on premises contact centers have been around for a really long time and I think a lot of companies are still doing on prem centers. If they want to do a lot of the innovation, they're having to go over the top with like a cloud based solution. So that may affect some of the sentiment that we're seeing now. The next one I'm going to talk about is CRM customer relationship management. Top providers for CRM, Agile, CRM, HubSpot, Microsoft and Salesforce, 7.90 was the top customer, I'm sorry, was the average customer sentiment score.
37% of companies are in the success group. So a little bit lower than we saw in the context, center side of things. Now, a few other companies did win customer sentiment only awards and that was Adobe Copper, Oracle, ServiceNow and Zendesk. And then those who had higher than average percentage of their customers in the business success category included freshworks and Monday.
Servicenow actually had the highest overall score for Sentiment and Microsoft had the highest overall score for business success. I would say in this category. In this category of CRM, most vendors had higher than average sentiment scores compared to others in the study. So it was a really, companies seem to be pretty happy with their vendors in this space. Ross.
[00:11:41] Speaker A: All right, so let me talk about CPaas now. So cpas for cpAs, we had the most much star top provider winners of the entire study. And those are eight x eight Aws, infobip, link, mobility, message, bird, Twilio and vonage.
And in terms of customer sentiment, those companies had an 8.08 on average. And Infobit, incidentally, had the highest customer sentiment rating of anybody in this study.
[00:12:16] Speaker B: Right? From all categories?
[00:12:18] Speaker A: Yes.
[00:12:19] Speaker B: Yeah. Okay. Yeah. Interesting. Okay. Yeah.
[00:12:22] Speaker A: As well as obviously then for cpas too.
And then customer, customer, the percentage of companies in the success group is 75%.
We also had one winner for customer sentiment only and that was Cisco and then Alibaba and out of China and Cinch were designated as winners for business success.
As I mentioned, Infobip had the highest customer sentiment rating and AWS had the highest percentage of customers in achieving business success.
Okay, next up is AI development platform and our top providers for awhile. Top providers for AI development platforms are AWS, Google, IBM and OpenAI.
They earned that for having a customer sentiment rating of 8.2.2 or better. And 37% of their companies fell in the success group.
For customer sentiment. Meta earned designation and we had no company that achieved business success had a higher than average business success number of companies in the business success category. But in terms of who performed well in this category, IBM had the highest customer sentiment rating score.
And actually the AI development platform, customer sentiment ratings were highest across the entire study here too.
Okay, bots, top providers of this, you know, very fragmented market. We had a lot of vendors that, you know, we didn't have enough ratings to count them individually. But in terms of those who we didn't have enough data for, we had Acera, AWS, Microsoft, OpenStream, AI and variant designated as top providers and they earned a customer sentiment rating of 8.18 or above. And then 50% of their companies were in the success group for customer sentiment. We had one additional provider that was Genesis, and Oracle is the one additional provider we had for the business success award.
Robin, off to you.
[00:15:01] Speaker B: All right, so agent assist, this is just such an interesting area these days, everyone's looking at agent assist. And so top providers for agent assist are Cisco, Google and Nice. So those three, 8.11 was the average customer sentiment rating. So they had to get at least that to be a top provider and at least 48% of their customers in the success group to be a top provider. We also awarded a customer sentiment only award to eight by eight talk desk and, and Zoom for agent assist. And then for business success, AWS, Genesis and Zendesk. Now, looking at the highest scores, Zoom actually had the highest score for customer sentiment in agent assist, which is pretty cool for Zoom, I think, too, because they've probably compared to other companies, one of the newcomers in the contact center space overall, in the CX space overall. And they have just like gone so quickly, you know, in adding innovations to their platform. So it was kind of cool to see that. And then nice was the highest for business success. So the highest percentage of customers for nice compared to others, you know, were in the business success group, so they were generating measurable success with the technology. Now, I will say that most companies actually do use, not all, but most use their CCAs provider for agent assist either directly, either the CCAs provider offers an agent assist that they internally develop or through a partnership between the CCAS provider and a third party agent assist vendor. But we did see a high percentage of companies using their ccats provider for this topic.
[00:16:40] Speaker A: Let me talk about self service knowledge management.
Always super critical, but really more so now with the arrival of generative AI. You have to have a strong knowledge management core if you're going to be assisting customers and giving them information that they can actually serve themselves as opposed to escalating to live agent. So for top providers there we have Bloomfire, nice, Salesforce and variant. And I should note, we actually, we didn't note this, but we're listing all of these companies in alphabetical order. Yes, this is an aside there.
And these providers earned an 8.09 customer sentiment score on average or better. And then a pretty high percent of companies fell within the success group, 82% or higher.
For customer sentiment, we awarded HubSpot, Liferay, Upland and Zoho. And then business success is, you know, as I said, a lot of companies fell in that success group. And here we had a good list of companies that earned business success designation. Egain guru, live agent, Servicenow and Zendesk.
When we look in terms of the ratings, Upland had scored the highest in customer sentiment and then Bloomfire had the highest percentage of companies in the success group, 96% of its companies got.
Yeah, that's a real lot.
[00:18:25] Speaker B: Real high number.
[00:18:26] Speaker A: Yeah. So. And that's highest across the study as well.
[00:18:31] Speaker B: Yeah. Okay, so next is voice of the customer. So this is obviously where you're using survey tools and analytics tools to measure what your customers are saying about you, whether it's a five star rating or one to ten rating or more importantly open ended ratings, where we can see the whys behind the whats. So the top providers there, nice question. Pro SMG and variant, average customer sentiment rating was 7.98, so they had to get that or higher. And 31% of customers were more in the business success group. For customer sentiment only, we saw higher than average scores from Alkimer, concentric.
I'm sorry, concentrics confirm it. And qualtrics. And then for business success, customer gauge in moment, Surveymonkey and Upland.
As far as scores go, nice earned the highest score overall for customer sentiment and question Pro had the highest percentage of its customers in our business success category.
Another category I wanna talk about is visual engagement. Now this is an area that's kind of interesting where we have our consumer research saying, well, the consumers really want to do more and more with visual engagement when they interact with companies they do business with. But the businesses don't make it real easy for them to do that, at least initially. Not initially. It's more of an, you know, an escalation path. So this is, you know, an area that we might want to be focusing on more as an industry.
But that said, the top providers for visual engagement, Adobe, Cisco, Genesis, Oracle and Zoom 7.98, customer sentiment is the average. So they had to do better, add or better from that and at least 80% of customers in there in their success. In our success group, it's interesting this year, by the way, we did see fewer like visual engagement specialty providers having enough ratings this year. We've seen it like glance and Glia and companies like that we saw in previous years. This year we didn't get enough ratings for them, if you're wondering. So it was more the larger platform providers customer sentiment award went to go to as well as Microsoft. Nobody got a business success only award in this area.
And then as far as our top scores in visual engagement, Genesis had the highest customer sentiment score and Cisco had the highest score for business success.
[00:21:02] Speaker A: Okay, let me wrap up with the final two. Workforce optimization.
The top providers here are Cisco, Envision, Nice, Talkdesk and Twilio.
These companies had an 8.0 customer sentiment score on average or higher. And then 74% of companies fell in the success group.
We awarded one additional provider that was five nine for customer sentiment and then two providers for business success. And they are Alveria and Genesis. In terms of the scores, Envision had the highest customer sentiment rating and top desk had the greatest percentage of customers in the success group.
And then lastly is workflow automation. Our top providers for workflow automation are Genesis, Monday and Nice. And they scored an 8.19 customer sentiment score on average or better and 74% rather percent of their customers are in the success group.
We had four additional companies earning designation for customer sentiment ratings and those are Dialpad, Salesforce, Zapier and Zoho. And then five providers for business success. And those are Cisco, Cognigy, Microsoft, Oracle and Variant.
When we look at scores overall, Zapier had the highest customer sentiment rating score and nice had the highest percentage of its customers in the success group.
And that wraps up the winners. Congratulations to all of these companies.
Robin, do you have anything else to share?
[00:23:02] Speaker B: You know, I guess I'll wrap up by just saying that there are a lot of companies. Ask me, well, how do we, how do we use this information? What do we do with it? And I think there's a lot of things that enterprise organizations can do with this information.
We can't tell you which is the provider for you, obviously, without a lot of back and forth and understanding what your needs are, what your roadmap is, what you're planning to do.
Anybody in the top provider designation, absolutely, for sure should be on your shortlist in any of these categories that you're looking at now. You also might wanna look at companies who are in the customer sentiment and business success categories, too, depending upon what's most important to you.
If your company is all about customer sentiment, we have gotta, you know, be working with somebody who's really going to, you know, be firing in all cylinders on the different ratings that we're looking at. But business success, yes, it's important to us, but we feel like we have a lot of control over that and we can handle that. So, you know, maybe you want to add those customer sentiment winners to your categories as well. We can drill down into all of these categories. We have actual ratings. So if things like, you know, customer service or technology features or security, those are the important things to do. We can tell you how each of these vendors scored, how they scored compared to others.
And we can really help enterprise organizations select the right technology and select the right technology partner based on, you know, not our opinion at all, based on just what customers are saying about these providers. So I just like to point that out. I think it's, it's really, you know, it's a, it's not like the only thing you'd look at when you're selecting some, you know, company, but it certainly is good tool to be using as you're evaluating providers.
[00:24:48] Speaker A: Okay with that, I think. Again, just want to thank, I'm sorry. Just congratulate all of our winners and thanks to all of our listeners for tuning in today.
If you're interested in learning more about our Metrocast service or any of our other research, we would be absolutely happy to hear from you. So please feel free to reach out to either Robin or myself via email or LinkedIn or contact us via the contact button on the Metrogy website at www.metrogy.com. That is all for now. On behalf of the Metrogy team, goodbye till next time and take care, everybody.
[00:25:28] Speaker B: Thank you.
