Everything you missed from Google I/O 2024: Gemini, Project Astra and more

At this week’s I/O, Google gave the developer community a glimpse of what it has in store for next year and, unsurprisingly, it’s all about artificial intelligence. The company’s lengthy keynote speech Tuesday focused almost entirely on AI and the upgrades the company has made to its Gemini model. CEO Sundar Pichai even noted that presenters said the word “AI” 121 times. Some features are currently rolling out while others will appear later this year after additional testing. But here are some of the most interesting things the search giant showed off in Mountain View.

Gemini 1.5 Pro and Flash

Google brings its Gemini 1.5 Pro model to Gemini Advanced, Google’s version of ChatGPT Plus, with an expanded pop-up of 1 million tokens. This will help him “make sense of multiple large documents, up to 1,500 pages in total.” And now you can download these files directly from Google Drive. For those who need the fastest possible response, Gemini 1.5 Flash is a smaller model “optimized for narrower or high-frequency tasks.” Both versions are available in preview today in over 200 countries ahead of an official launch in June. A 1.5 Pro version with a 2 million context window is also available to Google Cloud customers, but you will need to join a waiting list.

This Tweet is currently unavailable. It may be loading or has been deleted.

AI Insights

Google has spent the last year testing its Search Generative Experience (SGE), which places AI-generated answers at the top of search results and is intended to quickly answer your question at a glance; no more clicking on the blue links. At I/O, Google renamed SGE “AI Overviews” and says it “will be rolling out to everyone in the United States, with more countries coming soon.” This feature has put some search-focused publishers on edge since the data was revealed by AI. Previews are taken from their sites. But Google insists that “we see that links included in AI previews generate more clicks than if the page had appeared as a traditional web listing for that query.” Soon, Google says you’ll be able to adjust AI previews to get a simple or more complex answer.

Request photos

Even though your Google Photos account is neatly organized in Albums, it can still be a chore to get an image to appear when you need it. With Ask Photos, you can have a conversation with your photo albums through Gemini and the AI ​​will suggest photos for you. Google promises that you can go a little further than “show me photos of the Eiffel Tower” to ask things like “Show me the best photo of every national park I’ve visited” or “What themes have choose us for my child’s birthday parties? » Ask Photos is an experimental feature that will be rolled out “soon”.

Astra Project

Project Astra is a next-generation AI assistant that can visualize the world around you and provide you with tips and tricks throughout the day. In a demo (below), a Google employee accessed the AI ​​assistant on a smartphone, pointing its camera at various objects and asking Gemini to identify them or provide details about them. He identified the parts of a speaker, the type of code displayed on a monitor, and the neighborhood Google’s office was in by looking out the window. He also recalled where the employee had left her glasses. In our own demo, we got more mixed results, perhaps due to our artistic talents (video above). The company plans to roll out parts of Project Astra later this year through the Gemini app.

This Tweet is currently unavailable. It may be loading or has been deleted.

Veo video generator

Google revealed a new AI video generator, image maker and music creation toolkit during its keynote. Much like OpenAI’s Sora, the Veo video generator takes text prompts and turns them into 1080p resolution videos longer than 60 seconds. It is available today for “select creators in private preview” and via a waitlist. Google plans to release it more widely in the future, as well as on YouTube Shorts as a new video creation tool.

Gemini for Google Workspace Upgrade

Google added its Gemini intelligence to Workspace earlier this year, and its business-facing software will now get a Gemini 1.5 Pro upgrade. When people launch Gemini in the side panel of Gmail, Docs, Drive, Slides, and Sheets, Google says that “Gemini can answer a wider variety of questions and provide more relevant answers.” This includes asking Gemini to summarize long threads or Docs. The upgrade is available now for those using it through Labs or Workspace Alpha; everyone will receive it next month.

This Tweet is currently unavailable. It may be loading or has been deleted.

Hands-on with 3D videoconferencing from the Starline project

At I/O, Google showed off its Project Starline 3D video conferencing solution, and it’s pretty impressive. The Starline project leverages a powerful display called a light field display and six large web cameras that capture the user. Web cameras are designed to record 3D video of your face, body, and any objects you show during the call from different angles. You feel like you’re in the room with someone on a video call rather than in a small chat box. This won’t be something you use at home; your office will probably have to shell out to install Project Starline, but Google says the technology can increase attention span and memory. Discover our hands to find out more.

Scam call alerts

Google is testing an AI-powered alert system that warns Android users about fraudulent phone calls in real time. It leverages Gemini Nano, an AI model that can run locally on a smartphone without an internet connection. If the AI ​​detects keywords associated with a scam (like asking for a password), it may prompt you to hang up. You can close the pop-up if the call is legitimate. Details are scarce at this point, raising alarms among security professionals, but Google says Nano works locally, without an Internet connection. “This protection happens on the device, so your conversation remains private to you.” He promised more information “on this membership feature later this year.”

Android Theft Protection

Google is taking a cue from Apple with Android Theft Protection, which locks a phone if it’s been stolen to prevent thieves from accessing banking apps and other sensitive data on your phone. If your phone is seized, biometric authentication will be required and a delay before settings can be changed, which is intended to give you enough time to access another device and change your logins or erase the device. It will roll out via Google Play Services later this year to devices running Android 10+, with some features available in Android 15.

Excuse me, your phone is hacked

In an effort to prevent the proliferation of fraudulent apps on Google Play, a new feature tells developers if their apps are running on devices infected with malware. Among other things, this allows Android to notify a developer if another app might be recording the screen, overlaying content, or taking over the phone.

Google Cast comes to cars, starting with Rivian

Need to waste time charging an electric vehicle or waiting in the pickup line? Google Cast is coming to cars, starting with Rivian via a “simple over-the-air update…soon.” This feature only works when parked so as not to distract the driver. It’s also coming to other car brands with Android Automotive OS, but Google hasn’t specified which ones.

Everything announced at Google I/O