Google has announced the expansion of its experimental AI Mode, which now includes multimodal search. This feature allows users to ask questions about images they have uploaded or taken with their camera. The new image analysis functionality in AI Mode leverages the capabilities of Google Lens, enabling it to understand the entire context of a scene in an image, including the relationships between objects, their materials, colors, shapes, and positions.
AI Mode uses a “fan-out” technique, allowing users to ask multiple questions about an image and the objects in it, providing more detailed information than traditional Google search. For example, a user can take a photo of their bookshelf and ask: “If I liked these books, what other similar highly rated books are there?” AI Mode will identify each book and provide a list of recommended books with links for further exploration or purchase.
The new feature is already available for users of the Google app on Android and iOS. It not only allows users to get answers about the content of an image, but also to refine their queries, for example, by asking which of the recommended books is the shortest. According to Google representatives, expanding access to AI Mode will allow millions of users subscribed to Labs to take advantage of the new features without needing a Google One AI Premium subscription.
After a month of testing AI Mode, users have noted its convenient design, fast response time, and ability to understand complex and nuanced questions. The number of queries in AI Mode is on average twice as high as traditional Google Search queries, indicating its use for more complex tasks such as product comparisons, travel planning, and other research queries.
