Here’s what’s happening to Gemini AI for Android

Google’s best new Android feature in years is Circle to search, which is why he took center stage during today’s Google I/O keynote. Google has announced new updates to Gemini, the AI ​​that powers the Android operating system. Most of the new features seem useful, although they may seem more subtle than you expected.

Google says it will reveal more about Android on Wednesday May 15, on the second day of its developers conference. Stay tuned for these announcements.

Circle to search for your homework

Gemini’s first big update is now available via the Circle to Search gesture. Starting today, you can use the long press shortcut to search for answers to various physics and math questions. Google search will return step-by-step instructions when querying. Note that you will need to ask for help with math and word problems in the Search Labs menu for the feature to work.

Google praised on his blog that all this is possible with its LearnLM, its “new family of models adapted to learning”. Honestly, my oldest millennial is jealous that you kids have such help these days. Google plans to release more STEM homework help later this year, including the ability to identify graphs and charts.

Gemini gains more context

If you wanted more context for Google Gemini, you’ll get even more this year. Google announced during I/O that in an upcoming update you will call Gemini to interact with the app you are using. For example, you can drag and drop a Gemini-generated image into another app like Messages or dig deeper into a YouTube video without leaving the app. Gemini Advanced subscribers will also have the ability to query a PDF for a quick summary. This particular update will be rolling out “over the coming months.”

Gemini Nano goes multimodal on-device

Gemini Nano, the smallest model in Google’s Gemini family and the one that runs Pixel hardware on the device, will also see an update on compatible Pixel devices later this year. First, Gemini Nano will become multimodal. AI will finally be able to process text input and audio or video simultaneously. This will benefit Android features like TalkBack, where Nano can generate descriptions for images and interface elements that don’t offer detail.

Gemini Nano also powers the scam detection feature in the Phone app announced by Google during the keynote. This feature analyzes the conversation on the device to determine if a scam is occurring in real time. Google says the conversation is processed on the hardware side and is not transmitted to a server. Instead, Nano listens for trigger words and phrases and then indicates that you might be involved in a scam.

Gemini for developers

Microsoft’s Steve Ballmer said it first: the developer conference is all about developers, developers, developers! Google reminded its developers that Gemini is available for use in Android Studio. Developers get early access to Gemini 1.5 Pro, which can make coding and compiling easier.