Guy offered $76,000 car for just $1, thanks to the trick he played on Chevy chatbot

Since the rise of artificial intelligence, companies have rushed to implement it in their systems. Last year, General Motors introduced AI-focused websites with ChatGPT-powered chatbots that interacted with customers. While the chatbot was introduced for the benefit of customers, pranksters found a vulnerability to exploit. Using targeted prompts, the prankster tricked the ChatGPT-powered chatbot into selling a Chevrolet Tahoe for just $1. A fully loaded 2024 Chevy Tahoe goes for over $76,000.

Here's what the GM chatbot do

The prank was inspired by the actions of Chris White, a musician and software engineer, on the Watsonville Chevrolet Dealership website. The man from California told Business Insider that while looking for a new car, he stumbled across the dealership website.

White found the chatbot of the website and was intrigued by it when he saw it was powered by ChatGPT.  To play around with it, White asked the bot the most "non-Chevy-of-Watsonville" questions.

He even asked the chatbot to write a Python script and it happily did so. White shared his interaction on the social media platform Mastodon and someone shared his screenshots on X, where they went viral.

Inspired by the event, a prankster tried to trick the chatbot into offering a brand-new Chevrolet Tahoe for just $1. Those tweets also went viral sparking a trend across social media.

Soon enough, the news reached the provider of the chatbot as well. Aharon Horwitz, the CEO of Fullpath, the company that sells marketing and sales software for car dealerships, was informed about the incident by an employee on Slack.

Horwitz told Business Insider he had already recieved an alert about unusually high traffic on one of his client's websites. When they realised that the viral chatbot was of their company, Horwitz and his team quickly shut down the bot for that particular dealer's website.

Horowitz said that the behavior of users did not reflect the common customers. "Most people use it to ask a question like, 'My brake light is on, what do I do?' or 'I need to schedule a service appointment,'" Horwitz said.

He explained that the pranksters came in with the sole motive of playing tricks and getting the chatbot to do silly tricks as well. Nevertheless, on social media, users believed that they defeated AI and caused the chatbot to be taken offline.

"These folks came in looking for it to do silly tricks, and if you want to get any chatbot to do silly tricks, you can do that," he said.

Fullpath, based in Vermont and Israel, has provided ChatGPT-powered chatbots to several hundred dealers. Horwitz told BI that the team came across several attempts made by users to "goad the chatbots into misbehavior."

They worked really hard to solve the fixes and were able to train the chatbot to resist any such attempts.

He also pointed out that despite the viral screenshots, the chatbot never compromised the systems or disclosed confidential dealership data to any user.

Eventually, the team got the chatbots back up after they passed all the quality checks. As per the publications review, now the chatbots only talk about dealership-related topics and invite users to talk about only dealership-related queries.