At the I/O Developer Conference held earlier this year, Google introduced a new artificial intelligence milestone called Multitask Unified Model (MUM). This technology can simultaneously understand the information in various formats, including text, images, and videos, and draw insights and connections between topics, concepts, and ideas. Today, Google announced plans to integrate MUM in its own products, taking the lead in updating Google Lens visual search.
Google Lens is the company’s image recognition technology that allows you to use your phone’s camera to perform various tasks, such as real-time translation, identifying plants and animals, copying and pasting from photos, and finding items similar to those in the camera’s viewfinder, Get help with math problems, etc.
Google today said that it will use MUM’s capabilities to upgrade Google Lens to enable it to add text to visual search so that users can ask questions about what they see. In practice, this function can work like this. You can call up a photo of your favorite shirt in Google Search, and then click the Lens icon to ask Google to find the same pattern for you but on a pair of socks. By typing socks with this pattern (socks with this pattern), you can guide Google to find relevant queries. If you only use text input, it may be more difficult to do.
In another example, a part of your bicycle is broken and you need to search for repair tips on Google. However, you don’t know what this part is called. Instead of delving into the repair manual, you can point Google Lens at the broken part of the bicycle and type how to repair it. This allows you to directly connect to the exact moment in the video and thus helps.
Google sees these AI-driven initiatives as ways to make its products more helpful to end-users by enabling new search methods. By using mobile phone cameras as part of the search, Google’s goal is to remain relevant in markets where many of its core use cases are beginning to shift to other attributes.
In today’s Search On live event, Google announced that it will also use MUM in Google search and video search in other ways. The update to Google Lens will be rolled out in the next few months, noting that it still needs to undergo rigorous testing and evaluation as part of every new artificial intelligence model it deploys.
