Moveo’s LLM vs GPT-4 for Customer Experience

NEW MILFORD, N.J.--(BUSINESS WIRE)--Jul 24, 2024--

Moveo.AI announced that after rigorous comparison, its custom LLM tuned for CX outperforms GPT-4-0613 in all grading dimensions, except Markdown, where GPT-4 performs better. The evaluation was based on a random sample of hundreds of entries from Moveo’s production data, which neither our LLM nor GPT-4 had encountered before. Each entry was converted into a prompt consisting of the user question, conversation history, grounding knowledge from the collection documents, live instructions, and custom instructions.

This press release features multimedia. View the full release here: https://www.businesswire.com/news/home/20240723855013/en/

As can be clearly seen in this table, Moveo’s custom LLM outperformed GPT-4 in four critical dimensions that are the cornerstone of a great Customer Experience: Hallucination, Repetitions, Disambiguation, and Readability. The two models are equal in Language while GPT-4 performs better only in Markdown use. (Graphic: Business Wire)

Methodology

The grading process assessed Moveo’s LLM and GPT-4 responses across 8 dimensions that capture critical traits within the CX setting:

• Hallucination
• Repetition
• Disambiguation
• Live agent handover
• Language
• Markdown, and
• Latency

Each dimension received a score, determining which LLM provided a better response. To evaluate the performance of the different models, Moveo used a separate GPT-4 instance as a “grader,” performing a single API call for each of the samples.

Results

Moveo’s custom LLM outperforms GPT-4-0613 in all grading dimensions, except in Markdown, where GPT-4 performs better in stylistic formatting. Most importantly, it is worth mentioning that in terms of hallucination, GPT-4 performs worse, which could hurt Customer Experience. For example, if GPT-4 provides incorrect information about a product, it could lead to potential liabilities, customer dissatisfaction, and increased support requests.

Moveo’s LLM responds in only 5 seconds, while GPT-4 takes at least 18 seconds. In that time, Moveo.AI could have handled more than 4 inquiries, significantly enhancing support efficiency and customer satisfaction.

According to Panos Karagiannnis, CEO of Moveo.AI, “Enterprises need vertical-specific LLMs as every customer interaction is an opportunity to build trust and loyalty. By minimizing hallucinations and connecting to real-time information systems, our LLM significantly beats GPT-4, reduces the risk of customer dissatisfaction and potential liabilities, and sets a new standard in CX”.

To learn more about Moveo’s proprietary LLMs, please visit: https://moveo.ai/

About Moveo.AI

Moveo.AI is a Conversational AI platform transforming how enterprises interact with customers. Moveo’s LLM, trained on historical and real-time CX data, powers GenAI agents to seamlessly connect to real-time data and unstructured knowledge bases to provide accurate and contextually relevant answers to inquiries.

View source version on businesswire.com:https://www.businesswire.com/news/home/20240723855013/en/

CONTACT: Moveo.AI

Panagiota Gkotsi

+306981928344

panagiotagkotsi@moveo.ai

KEYWORD: UNITED STATES NORTH AMERICA NEW JERSEY

INDUSTRY KEYWORD: DATA MANAGEMENT APPS/APPLICATIONS TECHNOLOGY SOFTWARE NETWORKS ARTIFICIAL INTELLIGENCE INTERNET

SOURCE: Moveo.AI

PUB: 07/24/2024 09:10 AM/DISC: 07/24/2024 09:10 AM

http://www.businesswire.com/news/home/20240723855013/en