Microsoft's latest Copilot updates include a mobile version of the multimodal Vision tool
Microsoft just announced several updates to its Copilot AI assistant, and some sound downright useful. It’s bringing Copilot Vision to mobile, but with some new features. For the uninitiated, this software originally launched for the Edge web browser and gave Copilot the ability to “see” and comment on the contents of websites.
The company is upping its game for the mobile version, adding some multimodal functionality. It’ll be able to integrate with your phone’s camera to “enable an interactive experience with the real world.” Microsoft says it can analyze both real-time video from the camera and photos stored on the device
Microsoft gives an example of Copilot Vision analyzing a video of plants to determine if they are healthy or not and suggesting actions to take. We’ll see if it can actually perform that kind of nuanced reasoning. Modern AI companies love to promise the world and then, well, you know the rest. In any event, the mobile version of Vision is available today in the Copilot app for iOS and Android. The web version is also coming to Windows.
Microsoft is bringing Copilot Search to Bing to “seamlessly blend the best of traditional and generative search together to help you find what you need.” The company is now calling Bing “your AI-powered search and answer engine.” Like most AI web search tools, this provides summaries to answer queries.
Microsoft says this can take the form of a simple paragraph, like Gemini AI for Google searches, but that it also can provide “images and data from your favorite publishers and content owners.” Copilot Search is rolling out today.
The company also introduced something called Copilot Memory. This is Microsoft’s attempt to bring more personalization to Copilot. After all, it’s tough to have a true AI companion when it doesn’t remember anything about you. With this addition, Copilot will be able to remember specific details about your life, like “your favorite food, the types of films you enjoy and your nephew’s birthday and his interests.”
The company touts that the software will recommend actions based on what it remembers. To that end, Microsoft says Copilot will be able to do stuff like buy tickets to events, order flowers and make dinner reservations. It says the service will “work with most websites across the web.” We’ll see how that works out.
The update brings some other tools to the table, like the ability to auto-generate podcasts based on specific topics and offer shopping advice based on sales history across the web. These updates begin rolling out today, but it may not hit every user for a bit. Microsoft says availability will expand in the coming weeks and months.This article originally appeared on Engadget at https://www.engadget.com/ai/microsofts-latest-copilot-updates-include-a-mobile-version-of-the-multimodal-vision-tool-182752162.html?src=rss