Google Lens search with video and voice is the coolest AI feature yet

Google has added a suite of new generative AI-based features to its increasingly artificially intelligent search engine.

In a blog post today, Google unveiled the ability to search via videos you’ve captured on your phone within the Google Lens feature, while simultaneously asking questions about what you’re recording.

From here, Google will process what you’ve asked in the context of what’s on screen and produce an AI Overview, as well as some helpful web links. it’s live today in the Google app for iOS and Android, but you’ll need to enroll in the “AI Overviews and more” experiment in Labs to take it for a spin.

Google explains: “Say you’re at the aquarium and want to learn more about some interesting fish at one of the exhibits. Open Lens in the Google app and hold down the shutter button to record while asking your question out loud, like, “why are they swimming together?” Our systems will make sense of the video and your question together to produce an AI Overview, along with helpful resources from across the web.”

As well as video, Google is adding the ability to ask voice questions when you take a photo in Lens. Google says users will be able to point the camera, hold the shutter and ask a question while doing so. This feature is fully live in the Google mobile apps.

There’s also an update that’ll improve the results page when you’re using image search to match items you see in the wild with shopping options. Lens can now identify exact products, rather than just something that looks like the image you’re capturing.

Circle to Search – one of the cooler AI features – is getting another upgrade that’ll identify any music playing within the content on your screen. So, if there’s a song in a tweet you’re trying to discern, for example, Google will name that tune.

Finally, for US users, Google is rolling out new tools to organise search results pages with AI.

“This week, we’re rolling out search results pages organised with AI in the U.S. — beginning with recipes and meal inspiration on mobile. You’ll now see a full-page experience, with relevant results organized just for you. You can easily explore content and perspectives from across the web including articles, videos, forums and more — all in one place,” Google writes in the blog post.