Google explains what on-device AI is for Android

On-device AI is a big priority for Android going forward, and Google shared more development resources at I/O 2024.

The I/O 2024 session “Android On-Device AI Under the Hood” provided “good use cases” for on-device generative AI:

• Consume: Provide a summary or overview of the text
• Create: suggest replies in messaging applications or generate/reform text
• To classify: Detection of feelings/moods in conversations or SMS

In general, the benefits include secure local processing, offline availability, reduced latency, and no additional (cloud) costs. The limits are a parameter size of less than 2 to 3 billion, which is “almost an order of magnitude smaller than cloud-based equivalents.” There is also a smaller popup and the model will be less generalized. As such, “fine tuning is essential to achieve good accuracy.”

Gemini Nano is Android’s “fundamental mode of choice for creating on-device GenAI replications”, but you can also run Gemma and other open models.

So far, only Google apps – Summarize in Pixel Recorder, Magic Compose in Google Messages, and Gboard Smart Reply – take advantage of it, but Google is “actively collaborating with developers who have compelling use cases on the Gemini device » via an early access program. . These are expected to launch in 2024.

Meanwhile, Google will soon use Gemini Nano for TalkBack captions, Gemini dynamic suggestions and spam alerts, while a multimodal update will come later this year “starting with Pixel”.

Google also noted the state of on-device generation AI a year ago and improvements since then, such as hardware acceleration:

Learn more about Android 15: