The CHRO of IBM reveals what went wrong with their initial AI chatbot rollout—and how they shifted strategies

In the footrace to roll out AI within the workforce, companies are bound to slip up along the way.

IBM has been open about its dedication to build, implement, and scale company-wide AI, and over the past decade the company has built 280 different AI mechanisms within HR alone. But that doesn’t mean the tech giant hasn’t had a few hiccups getting there. When the company first introduced its benefits assistant chatbot, AskHR, to staffers back in 2017, it was not a runaway hit.

“When we started on this journey, we started on it as a technical change. ‘Here’s this technical tool,’” Nickle LaMoreaux, chief human resources officer at IBM, tells Fortune. “And what happened was nobody used it. The technology was there, the tool was there, but behavior wasn’t there.”

She says staffers weren’t ready for that change back then—especially when the company’s strategy was more calculated than cordial. When IBM realized its workforce wasn’t going to start using AskHR willingly, leaders brought down the hammer. In 2018, the company told more than 20,000 workers that they were spending too much time on busywork, and that rather than ask the HR department questions as usual, they would have to use the chatbot instead.

That didn’t make staffers happy. Before that moment, IBM’s HR department had an employee CSAT score—which measures an individual’s satisfaction—of +19, on a scale from -100 to +100. But within that first year after the firm required workers to use AskHR, HR’s CSAT rating plummeted to -35.

That was a wake up call.

“We were trying to force the behavior change, but we thought about it from the perspective of the HR team,” says LaMoreaux. In the pursuit of eliminating busywork, they force fed employees a tool they weren’t comfortable with. “There’s a manager on the other end that says: ‘You want me to type into a chatbot?’ And again, this was 2018. So we started asking managers directly: ‘How could the chatbot be better? What do you like about it?’”

What staffers most enjoyed about AskHR, according to LaMoreaux, was that they could ask questions in real time and with customized responses. The chatbot could automatically synthesize a worker’s career data to point them to the policies that were most relevant to them. Previously, they would have to sift through dense employee manuals to find what they’re looking for. After listening to what employees really wanted, LaMoreaux says IBM tweaked AskHR to make everything even more direct, instantaneous, and personalized. The company developed the chatbot to create more concise and digestible answers, assist job transfers to get staffers into their desired positions quicker, and roleplay nerve-wracking conversations like performance evaluations.

“By taking that feedback from them, being centered to who your end user is, that’s what then got us out,” she says, referencing HR’s then-low employee satisfaction score. “And now our score is consistently high.”

Today, the CSAT score for the HR team is in the +80s—a huge leap from the worker dissatisfaction it grappled with before. AskHR has now evolved to handle 94% of employee queries and resolves around 10.1 million interactions per year for the HR department, according to the company. And while it handles the vast majority of questions, LaMoreaux says that the chatbot will direct workers to arrange a meeting with an HR leader directly if their issue is more sensitive. Situations like low performance or misconduct are still handled by humans.

Although many businesses rolling out AI tools in 2024 emphasize the importance of having a human touchstone, LaMoreaux says it was initially hard to see how people fit into the tech equation. She emphasizes that user feedback and change management in employee behavior, like asking workers to automate some of their duties gradually rather than all at once, is crucial to successful AI implementation. She warns that simply throwing chatbots or copilots into a workforce and expecting them to adjust only builds resentment against the new tech and leadership team.

For companies just now adopting their own tools, it may seem daunting to establish a new way of work optimized by AI. But LaMoreaux says HR leaders shouldn’t be afraid to break the mold—that’s what drove the business and employee gains IBM couldn’t have tapped into before. Thus far, AskHR has automated 765,000 tasks and trimmed down the HR operating budget by 40%.

“Most people say don’t experiment in HR,” she says. “But we are now moving our AskHR to generative AI. That allows us to trade faster, the natural language is a lot better, and it’s giving us better outcomes.”