Google is introducing a new multimodal search capability to AI Mode, which is an experimental feature in Google Search allowing users to pose complex, multi-part questions and follow-ups for deeper insights. This update empowers users who have access to AI Mode to inquire about images they have uploaded or taken with their camera. The enhanced image analysis feature in AI Mode leverages the multimodal capabilities of Google Lens. According to Google, this functionality can comprehend an entire scene within an image, understanding the relationships among objects, as well as their materials, colors, shapes, and arrangements.
Through a technique known as “query fan-out,” AI Mode can generate multiple inquiries regarding the image and the items it contains, offering more nuanced information than standard Google searches. For instance, if a user takes a photo of their bookshelf and asks, “If I enjoyed these, what are some similar books that are highly rated?” AI Mode will identify the titles of the books and provide a curated list of recommendations, along with links for users to explore further or make purchases. Additionally, users can refine their search with follow-up questions, such as, “I’m looking for a quick read.
Which one of these recommendations is the shortest?” During this week’s announcement, Google revealed that AI Mode will now be accessible to millions more users who are part of Labs, which is Google’s platform for experimental features and products. Initially, AI Mode was exclusive to Google One AI Premium subscribers. By launching this feature, Google aims to compete with popular services like Perplexity and OpenAI’s ChatGPT Search.
The company has indicated plans to continue enhancing the user experience and expanding the capabilities of AI Mode.