A CMO tried Google's AI tools and stopped using many of them

• The executive used Google's Performance Max, Smart Bidding, and Gemini.
• They found these AI services sometimes disrupted strategies or performed no better than humans.
• Google remains optimistic, adding new AI-driven ad features.

Before another $100 billion or more is spent on AI chips and data centers , it may be wise to pause and consider how strong demand will be for the end products of this next wave of computing.

There are already several AI-powered tools and services being used by paying customers. How are these performing? If they don't live up to the hype, then this unprecedented investment cycle may result in " pretty woeful " economics, as Amazon CEO Andy Jassy suggested on an earnings call.

Lately, there are signs that all is not well when it comes to the real-world performance of artificial intelligence. A chief information officer recently canceled an AI upgrade from Microsoft. AI-powered customer service has been " underwhelming " so far, a software engineer who used Klarna's AI agents said.

I checked in recently with a chief marketing officer who spends millions of dollars a year on digital ads and other technology. This person is a pragmatic tech optimist and early adopter.

This person's company has been trying out several Google AI services and tools. They shared some of the results with me, while asking to remain anonymous so they could speak honestly about their findings. Their identity is known to Business Insider.

Performance Max

First, the executive's company used Google's Performance Max service. This uses AI to automatically determine where ads should run across Google sites, including Search, YouTube, and Display.

For one major part of their business, the chief marketing officer found that Performance Max disrupted the advertising strategy so much that they stopped using it.

Other marketers have had trouble with Performance Max , BI's Lauren Johnson reported in late 2023. Some said they couldn't control where Google's AI decided to place ads, which made it harder to track and measure ad spend.

Google told BI that advertisers got more conversions and other value when using Performance Max.

Smart Bidding

The chief marketing officer said their company also tried Google's Smart Bidding tool, which uses AI to automate and optimize bidding in Google's real-time ad auctions.

They said that this tool turned out to be no better than just having their company's human advertising employees bid on Google ads, so they went back to that.

"You can't outperform a human" was the executive's summary of this situation.

Google said advertisers were in charge of how they used Google ads. Adjusting a budget, performance targets, or keyword bids can help create a more cost-effective campaign, it added.

Gemini for marketing emails

Third, the chief marketing officer said the company tried using Google's Gemini AI service to automate marketing emails. Roughly 60% of the time, the emails were great.

But 40% of the time, the emails were "unusable," they said. As in, so bad that they did damage to customer relationships. So the company had to use human employees to spot-check for this 40% problematic set of emails and separate out the bad from the good.

This made "the time and effort savings worthless," the exec said. So they scrapped that, too.

"Using generative AI to automate email content has not panned out (yet)," they added.

Google describes Gemini as a "good starting point" if you're looking for writing ideas. You can even get a choice of different drafts of the AI model's response so you can pick the best starting point.

"AI can be a great tool for enhancing human creativity, but can never replace it," the company said.

Google is still excited

This is just one example. Other top marketing executives likely have had different AI experiences.

Philipp Schindler, the chief business officer at Google, said last week during an earnings call that he continued to be excited about what this AI era was bringing to the ad industry.

"This quarter, we announced over 30 new ads features and products to help advertisers leverage AI and keep pace with the evolving expectations of customers and users," he added.

For example, Google added AI-driven profit-optimization tools to Performance Max and its standard shopping ad campaigns. Advertisers that used this profit-optimization technology alongside Smart Bidding saw an average 15% uplift in profit compared with revenue-only bidding, he said.