‍

HomeServe is a leading peace-of-mind provider for nearly 5 million homeowners in North America. Working with 1,300 utilities and municipalities, and 2,500+ contractors, their mission is to free homeowners from the worry, inconvenience, and expense of home emergency repairs. Fast, thorough, always-available service is core to their operation.

In 2020, HomeServe built Charlie, an automated voicebot, to deflect more straightforward, easy to address calls from contact center agents and provide customers faster self-service resolutions. Crucial to the program’s success was the voicebot’s ability to recognize the callers’ intention–many of whom are calling with emergency repairs–and triage the call to the correct department.

‍
The Problem

Working between Excel and coding notebooks, HomeServe didn’t have the right tools for their conversation designers and customer experience leaders to develop, measure, and manage the voicebot.

The Challenge of Solving Invisible Problems

The project began with a lean team of customer experience leaders and Java engineers. Once the initial voicebot was live, they needed to monitor and improve its performance to ensure the automated service met the standards of agent-managed calls.

Before working with HumanFirst, the team did this manually; analysts read as many transcripts as they could, looking for signs of underperformance. From the beginning, the volume of conversations was unmanageable. Auditing and developing the voicebot would be impossible at scale.

"We were using an incorrect system to manage our bot at that size, and it was growing. Out-of-the-box tools weren’t working for us. Ultimately, we would export the model and work with it in Excel. We were looking at a lot of word clouds and struggling to audit performance."

Victor Alejo

Automation Engineering Manager

The Technical Divide

As the project sprawled, maintenance became increasingly technical. The voicebot was built and managed on prohibitively technical platforms, making it difficult for the customer service and conversation design teams to collaborate and improve the model with more data coming in.

"We knew we wanted language experts and customer service specialists to lead the project, but those people don’t always come from technical backgrounds. We needed a way to keep them in charge."

Ross Parkes

New Capabilities Lead

‍

The Solution

A Non-Technical Platform for Conversation Data

With the goal of improving the voicebot’s metrics, HomeServe was vetting third parties and attending conferences when they found HumanFirst.

"Compared to other solutions we were looking at, HumanFirst is very simple to use. The user experience is business friendly, where other tools, for nontechnical people, were much more complex."

Ross Parkes

New Capabilities Lead

With ready integrations to Dialogflow ES, where the initial voicebot lived, HomeServe could instantly import their existing NLU model and their conversation histories into HumanFirst to run analytics. Immediately, the team could identify confused intent categories, ambiguous call drivers, and missing training phrases.

Building with Transparency

HumanFirst provides a low-code, collaborative platform that makes it easy for business analysts, conversation designers, and automation engineers to apply AI to ground truth conversation data in order to develop, maintain, and improve their model’s performance.

With user-led search, semantic similarity, adaptive clustering, embeddings on upload, and access to any LLM, the data work that once required a technical team becomes available to any contributor. In an intuitive, drag-and-drop interface, business analysts and conversation designers can interact with the data to find patterns, disambiguate intent categories, add training phrases, and run tests to measure their progress.

"All of the sudden, our business analysts and conversation designers could see every conversation, run evaluations, and return with recommendations. It accelerated and clarified the process for the business analysts; they’ve really enjoyed using it."

Ross Parkes

New Capabilities Lead

"In a short time, I understood this would give us the transparency we needed. The voicebot became something our team could see and debug easily. Meeting our goals no longer meant guessing. We could see it all."

Victor Alejo

Automation Engineering Manager

A Business Case for a Better Solution

HomeServe used HumanFirst to review their existing bot from top to bottom. Their Dialogflow ES model had become big, messy, and hard to handle, with over 1,000+ intents. It was clear that simplifying the bot and migrating to Dialogflow CX would help them manage more complex conversations.

"The HumanFirst team was instrumental in helping to sell this transition internally. The data clearly showed the expected ROI: we’d see an immediate 5-10% gain in performance just by improving the intent classification. Without that data, I don’t think we could have made the case to invest; we would have stayed on the platform we were on."

Ross Parkes

New Capabilities Lead

"It’s not just the platform HumanFirst provides, but also their expertise. They brought themselves into our team, understood our process, and helped us create our ideal customer experience."

Victor Alejo

Automation Engineering Manager

‍

The Result

Migrating from Dialogflow ES to Dialogflow CX

Beginning with 1,000+ intents, the team transitioned to Dialogflow CX and simplified their model to 140 clarified intents. The initial project, which was four years in the making, was migrated to Dialogflow CX in only four weeks. The full transition was complete in three months, leaving HomeServe with a 90% simpler classification model.

"This was a massive step, especially since we had a hard deadline. Without the intent classification model we built in HumanFirst, it would have taken us another year to migrate and rebuild the model as it was. We’re gaining velocity across development, maintenance, measuring, and optimizing–all of that is revenue in the end."

Ross Parkes

New Capabilities Lead

HumanFirst integrates with Dialogflow CX to make continuous conversation monitoring and model improvement simple. The HomeServe team, now consisting of 7 hands-on analysts and conversation designers, is able to oversee and optimize every conversation.

All Non-Technical Hands On Deck

With HumanFirst, HomeServe achieved their primary goal of removing technical barriers, empowering language experts, customer experience specialists, and conversation designers to take the lead. Managing less intents with more precision and accuracy has made developers’ work more enjoyable.

"As a manager, I have a really specific goal: make my team happy. When you make these kinds of changes–simplify the process, improve collaboration, make the metrics visible–you’re making people happier. My team is less stressed, more confident, and better able to achieve their goals."

Victor Alejo

Automation Engineering Manager

Embedding Generative AI into Day-to-Day Processes

HumanFirst makes it easy to leverage LLMs with conversation data to generate call summaries, extract custom insights, generate training phrases and knowledge base articles, and understand data at scale.

"With the latest industry changes in AI, we’re excited to lean into the AI features to streamline our team’s work. HumanFirst is in our company to stay. I don’t have a doubt about that."

Victor Alejo

Automation Engineering Manager

‍