Google I/O 2024 top highlights: Major upgrades to Gemini 1.5 Pro, new models, Gen AI for search, and more

Imagine if everyone could have their own AI tutor. This idea is frequently discussed in Silicon Valley. LearnLM, a new family of models based on Gemini and fine-tuned for learning, aims to make this a reality.

SynthID is a watermarking tool for AI photos from Google. Starting today, it is being expanded to text and video modalities. 

A look at Google's AI principles:

Gemma gets several updates, along with introducing Gemma 2.0, the next generation of open models. Gemma 2.0 features a new architecture designed for exceptional performance and efficiency and will come in various sizes. 

The Gemma family is also growing with the addition of PaliGemma, Google’s first vision-language model inspired by PaLI-3.

Video frame extraction, parallel function calling, and context caching are all integral features, emphasising the importance of long context. This advanced functionality is set to launch next month. Meanwhile, Gemini 1.5 Pro and Flash are available today globally in over 200 countries and territories.

Later this year, Gemini Nano with Multimodality will be coming to Pixel. Gemini Nano with Multimodality is not the same as the Gemini 1.5 Pro in Gemini Advanced.

Dave Burke explains how Gemini is now "context aware" on Android. He fires up Gemini in a chat thread to generate an image related to what he's talking about with a friend. One can ask Gemini specific questions about videos one is watching.

This is a once-in-a-generation moment to reinvent what phones can do. We are reimagining Android with AI at the core. Google is unlocking on-device AI, introducing AI-powered search at your fingertips, and Gemini, your new AI assistant, is coming to Android smartphones. The update will allow Gemini to better use on-screen data, helping you make sense of information as you go about your day.

Hsiao explains that with Gemini Advance, your trip planning experience is streamlined, incorporating search, shopping, and maps. It can generate a customised itinerary based on your preferences and activities.

Sissie Hsiao explained the concept of personal experts, or "Gems," in the Gemini app. Gems can be tailored to meet your specific needs, which seems similar to OpenAI's Custom GPT. Further explaining, Hsiao mentioned that Gems are a great timesaver when you have a specific way you want to interact with Gemini.

Chip is the Gemini-powered teammate. In the demo of this AI Teammate prototype, it has a description (instructions) given by the user. Chip, the virtual teammate, is in your Google Chat with colleagues and it jumps into action as co-worker when needed. 

Aparna Pappu gives a demo on how Gemini can help you organise and track receipts in the side panel by extracting information from your inbox and entering it into a new spreadsheet.

Google is integrating its latest AI, Gemini 1.5 Pro, into the right sidebar of Workspace apps like Docs, Sheets, Slides, Drive, and Gmail. This virtual assistant will have access to all your saved information, providing support across all these applications.

In a fun demo, the speaker showcased the capabilities of AI overview. With this users can ask Google Search to plan something. Rose Yao shows troubleshooting an issue with a record player. Google Search gives an instantaneous AI Overview for troubleshooting the issue. Yao said that Gemini breaks down the video frame by frame, and Search is able to identify the make of the record player, then comb the web to find relevant info on how to fix it.

Introducing generative AI to Search, Sundar Pichai said, “Google Search is generative AI at the scale of human curiosity.” The Gemini model has been tailored specifically for Search and AI overviews. This new Gemini model, customized for Google Search, integrates the AI-powered chatbot's advanced capabilities and can help piece together all the information you need. Meanwhile, AI overviews let Google do the information seeking.

"Google will do the googling for you," Reid said, referring to the new Gemini model customized for Search.

Sundar Pichai has just introduced the sixth generation of TPUs, named Trillium, which offers a 4.7x performance improvement. Trillium will be available to Cloud customers in late 2024. Additionally, the Axion processor, featuring a custom ARM-based CPU, is also coming. Google also announced that the NVIDIA's Blackwell GPUs are set to be available in early 2025.

Google announces Veo, its generative video AI model. The model can create 1080p video from text prompts in different cinematic styles that can be edited using prompt. This seems to be Google’s answer to OpenAI’s text-to-video model Sora. The model will be on a platform called VideoFX.

Imagen 3 is an AI model that promises enhanced photorealism and more intricate details. According to Doug Eck, one will even be able to count the whiskers on a wolf's snout. It will also interpret prompts in a more natural, human-like manner. Sign-ups for Imagen 3 begin today on ImageFX and will be available soon for developers and enterprise customers.

Google announces Project Astra. Demis says it is meant to be a universal agent for everyday life and the reason why Gemini will be multimodal. Astra is "a universal AI agent that can be truly helpful in everyday life." Demis said that the pace and quality of interaction with Astra feel natural.

Gemini 1.5 Flash is a lightweight model optimised for tasks where latency is crucial. It is designed to be faster and more cost-efficient compared to the Pro model. Both models support up to one million tokens. Gemini 1.5 Flash is engineered for speed and cost-efficiency at scale.

CEO Sundar Pichai introduces Agents. Says, "Making AI helpful for everyone" is Google's "ultimate" goal. The CEO thinks AI agents are a big next step toward that.

In the demo, Josh Woodward showcases audio overviews, allowing you to compile a personalised audio guide from a collection of files.

Gemini 1.5 Pro, with one million tokens, is now available for all developers and consumers. The tech giant has increased the context window to two million tokens for developers in private view. Meanwhile, Gemini Advanced, with the one million token context window, is now available in 35 languages. Yes, I have already lost track of all these names. Gemini 1.5 Pro is now available on Workspace Labs

Google's new feature, Ask Photos will roll out this summer, with additional capabilities to follow. The feature lets Google Photos answer questions like "show me how my daughter's swimming has progressed" by searching for photos and creating a collection using Gemini. This deeper level of photo search promises to be helpful according to Pichai.

"We want everyone to benefit from what Gemini can do," said Pichai.

Pichai welcomes attendees with his casual humor. The CEO said that Google has been investing in AI for over a decade. Pichai said that the company sees many opportunities ahead. He elaborates on Gemini Era, the central theme of Google I/O 2024, and explained the uses of Gemini.

With an insightful montage of technological evolution pioneered by Google, the Google I/O 2024 has commenced.

An interesting scene is at play as the crew performs the tech check to ensure everything is in place ahead of the much-awaited keynote by CEO Sundar Pichai.(Image: Anuj Bhatia/The Indian Express)  

As we await the start of Google I/O 2024, attendees are gradually arriving and finding their designated seats. This time last year, Google I/O saw an overwhelming number of participants from around the globe. We hope to see similar crowds this year too. There seems to be a palpable excitement in the air for the keynote session, announcements, and interactive demos that are about to unfold.

Tech visitors began thronging the venue in Mountain View, California. More and more people are expected to join.

With just a few more minutes to go for Google I/O, users around the world are eagerly looking forward to the big announcements scheduled for today. Here is an image from the venue, which is gearing up to embrace thousands of Google fans, developers, young tech geeks, and more. Stay with us for in-depth coverage of the highly anticipated tech event of the month.

Ahead of the Google I/O 2024, the tech giant has announced that it is bringing Project Starline out of the lab and partnering with HP to begin commercialising the experience. 

Project Starline, introduced in 2021, is a technology that creates a "magic window" experience, allowing people to interact as if they are in the same room despite being miles apart. Using AI and 3D imaging, Starline augments remote communication, making meetings feel more immersive and in-person. After extensive testing at Google offices and with enterprise partners, it was found to improve attentiveness, memory recall, and overall presence.

After nearly three years, Google is now moving Starline out of the lab, aiming to connect distributed teams and individuals in the workplace. Partnering with HP, the tech giant plans to commercialise the technology in 2025, integrating it with video conferencing services like Google Meet and Zoom. More information can be found at starline.google.

Google launched its AI model Gemini in December 2023. The model has been trained on various data types including images, video, audio, code, and text. Ahead of the Google I/O 2024, there have been rumours that Google has plans to integrate Gemini into numerous products, replacing Google Assistant on Android devices.

Some reports also suggested that there would be a next-generation Assistant, dubbed "Pixie," which may be showcased at Google I/O. This may likely debut on-device with the Pixel 9 this fall. Despite the Pixel 8a announcement, Google I/O might still feature hardware previews, as seen in past events. The keynote is expected to highlight AI's role in enhancing Google's search business, with features like Circle to Search developed with Samsung for easier mobile search, potentially expanding to other devices and platforms.

At the upcoming, Google I/O 2024, perhaps one of the highly anticipated feature announcements is that of the Android 15 mobile operating system. It is expected that new features on Android 15 would include the integration of Generative AI, such as a chatbot and an image generator. Other potential features include app archiving to save storage space and improved security measures. Click here to read more

The Pixel 8a which reminds one of the iconic iPod Nano, is Google's latest mid-range smartphone, blending premium design with advanced features. It boasts a soothing aloe green color, a sleek industrial design, and a 6.1-inch OLED display with a 120Hz refresh rate. Powered by Google's Tensor 3 chipset, it brings AI features like Best Take, Magic Eraser, and Audio Magic Eraser to a more affordable device. Click here to read more about our experience with the Pixel 8a.

Google is all set to tease some AI-powered features for its assortment of devices. The tech giant showcased some new AI capabilities of the camera app on Monday evening. Based on a tweet, it appears that the device is a Pixel phone. Much like the Vision capability on OpenAI's ChatGPT, the demo shared by Google showed a user asking the camera what it sees via a voice prompt. The AI can be heard replying or describing what is in its viewfinder with impressive accuracy.

One more day until #GoogleIO! We’re feeling ?. See you tomorrow for the latest news about AI, Search and more. pic.twitter.com/QiS1G8GBf9

— Google (@Google) May 13, 2024