Google launches Guided Vision – visual search feature in Gemini Live mode

Google has significantly expanded the capabilities of its Gemini AI assistant with the launch of Gemini Live mode, which can analyze camera images in real time. This mode is now supplemented with the following features guide visionwhich provides search for objects in the frame.

Image source: blog.google

Gemini Live mode’s guided visuals debuted in September’s Android Drop update, and Gemini can also remember the location of items in Find Hub and integrate Google Keep lists in Messages. Guided visual aids Gemini Live describe in detail what’s on the screen. In the example presented by Google, a user points the camera at a shelf of spices and asks for instructions on where the oregano is – and the AI ​​assistant tells the user exactly where the spices are, even guiding the user’s hand.

This new feature is useful in many scenarios — not just finding household items. When a user is in a dimly lit restaurant, guided vision can be used to easily order from the menu. You can also use it to read the fine print on product labels. Google also offers tools to make the feature work – Gemini will tell you if you need to change the frame, move the camera or center the desired object.

Guided Vision will soon be coming to smartphones running Google Android 9 and above – the feature can be added to accessibility shortcuts.